5

When Upgrading to ruby 1.9, I have a failing test when comparing expected vs. actual values for a BigDecimal that is the result of dividing a Float.

expected: '0.495E0',9(18)
got:      '0.4950000000 0000005E0',18(27)

googling for things like "bigdecimal ruby precision" and "bigdecimal changes ruby 1.9" isn't getting me anywhere.

How did BigDecimal's behavior change in ruby 1.9?

update 1

> RUBY_VERSION
=> "1.8.7"
> 1.23.to_d
=> #<BigDecimal:1034630a8,'0.123E1',18(18)>

> RUBY_VERSION
=> "1.9.3"
> 1.23.to_d
=> #<BigDecimal:1029f3988,'0.123E1',18(45)>

What does 18(18) and 18(45) mean? Precision I imagine, but what is the notation/unit?

update 2

the code is running:

((10 - 0.1) * (5.0/100)).to_d

My test is expecting this to be equal (==) to:

0.495.to_f

This passed under 1.8, fails under 1.9.2 and 1.9.3

John Bachir
  • 21,401
  • 22
  • 137
  • 203
  • What does http://www.wolframalpha.com say is the result of the division? – sarnold Mar 10 '12 at 00:00
  • [(10 - 0.1) * (5.0/100) = 0.495](http://www.wolframalpha.com/input/?i=(10+-+0.1)+*+(5.0%2F100)) – John Bachir Mar 10 '12 at 00:13
  • 4
    Sorry but all bets are off once you introduce a plain float. If you use BigDecimals all the way (`(10 - BigDecimal.new('0.1')) * (BigDecimal.new('5.0')/100)`) then you'll get the result you're expecting. Your test is flawed and was only working by accident. – mu is too short Mar 10 '12 at 00:50
  • And have a look at the significant digits (`9(18)` vs `18(27)`), that's probably where the difference is coming from. If you chop the "got" off at 9 significant digits to match the "expected" then you'll get the "expected" result. – mu is too short Mar 10 '12 at 00:59
  • @muistooshort I knew all that (but it's good to see it spelled out), my question was mostly about if anyone had an idea about why the behavior would change with 1.8 vs. 1.9. – John Bachir Mar 10 '12 at 01:18
  • I've added a 1.8 vs. 1.9 comparison to my question. – John Bachir Mar 10 '12 at 01:21
  • If you want help with this you will need to provide the original calculation that produced the two different results. As it is, we can't reproduce your problem, because we have no idea what it is. – DigitalRoss Mar 10 '12 at 03:45
  • @DigitalRoss see update 2 above. – John Bachir Mar 10 '12 at 04:58

2 Answers2

17

Equality comparisons rarely succeed on FP values

The short answer is that the Float#to_d is more accurate in 1.9 and is correctly failing the equality test that should not have succeeded in 1.8.7.

The long answer involves a basic rule of floating point programming: never do equality comparisons. Instead, fuzzy comparisons like if (abs(x-y) < epsilon) are recommended, or code is written to avoid the need for equality comparison altogether.

Although there are in theory about 232 single-precision numbers and 264 double-precision numbers that could be exactly compared, there are an infinite number that cannot be so compared. (Note: it is safe to do equality comparisons on FP values that happen to be integral. So, contrary to much advice, they are actually perfectly safe for loop indices and subscripts.)

Worse, the way we write fractional numbers makes it unlikely that a comparison with any specific constant will be successful.

That's because the fractions are binary, that is 1/2 + 1/4 + 1/8 ... but our constants are decimal. So, for example, consider monetary amounts in the range $1.00, $1.01, $1.02 .. $1.99. There are 100 values in this range and yet only 4 of them have exact FP representations: 1.00, 1.25, 1.50, and 1.75.

So, back to your problem. Your result of 0.495 has no exact representation and neither does the input constant of 0.1. You begin the calculation with a subtraction of two FP numbers with different magnitudes. The smaller number will be denormalized in order to accomplish the subtraction and so it will lose two or three low-order bits. As a result, the calculation will lead to a slightly large number than 0.495, because the entire 0.1 was not subtracted from 10. Your constant is actually slightly smaller (internally) than 0.495. And that's why the comparison fails.

Ruby 1.8 must have been accidentally or deliberately losing some low order bits and effectively introducing a rounding step that ended up helping your test.

Remember: the rule of thumb is that you must explicitly program in such rounding for floating point comparisons.

Notes. To answer the question from the comments about simple decimal fraction constants not having exact representations: They don't have exact finite forms because they repeat in binary. Every machine fraction is a rational number of the form x/2n. Now, the constants are decimal and every decimal constant is a rational number of the form x/(2n * 5m). The 5m numbers are odd, so there isn't a 2n factor for any of them. Only when m == 0 is there a finite representation in both the binary and decimal expansion of the fraction. So, 1.25 is exact because it's 5/(22*50) but 0.1 is not because it's 1/(20*51). There is simply no way to express 0.1 as a finite sum of x/2n components.

DigitalRoss
  • 135,013
  • 23
  • 230
  • 316
  • 1
    Epic answer! In what sense do numbers like 1.01, 0.495,and 0.1 not have exact floating point representations? – John Bachir Mar 11 '12 at 00:39
  • 1
    I've added a note on this at the end of my answer. – DigitalRoss Nov 02 '12 at 19:08
  • The note on using FP as subscripts is dangerous since the range of contiguous integers, exactly representable in FP, is usually 8 to 9 bits shorter than for the corresponding integer type. – Hristo 'away' Iliev Nov 16 '12 at 08:40
  • Good answer about delta between BD on R1.8 and BD on R1.9. However, someone should have pointed out that sometimes scaled integer math is better and more efficient than extended precision floating point -- especially for financial data. – aks Aug 08 '13 at 04:26
1

See the Wikipedia article on floating point accuracy problems. It does a very good job of explaining why numbers like 0.1 and 0.01 cannot be represented exactly using floating point numbers.

The simple explanation is that these numbers, when represented in binary floating-point format, are recurring, just like one third is 0.3333333333... recurring in decimal.

Just as you can never represent one third exactly using a finite set of decimal digits, you cannot represent these numbers exactly using a finite set of binary digits.

DuncanKinnear
  • 4,241
  • 2
  • 31
  • 62