1

This is very strange to me:

irb(main):012:0> "100.7".to_f.modulo(1)
=> 0.700000000000003

Why the 3 at the end?

irb(main):019:0> "10.7".to_f.modulo(1)
=> 0.699999999999999

Same thing here...we are only getting the remainder of this value divided by one. It should be exact.

Tony
  • 17,359
  • 29
  • 118
  • 188
  • 2
    Duplicate of almost all the `[floating-point]` questions. See http://stackoverflow.com/questions/1089018/why-cant-decimal-numbers-be-represented-exactly-in-binary for example. – S.Lott Aug 18 '09 at 21:14
  • 2
    This is also a duplicate of about 100 threads on the ruby-talk mailinglist, or indeed any discussion forum of any programming language that has ever existed. – Jörg W Mittag Aug 18 '09 at 23:50

5 Answers5

13

What Every Computer Scientist Should Know About Floating-Point Arithmetic

sepp2k
  • 341,501
  • 49
  • 643
  • 658
3

This is typical floating point rounding. You simply cannot express every single decimal number in the fixed number of bits in a Float, so some values are rounded to the nearest value that can be represented.

Due to this, it is suggested that you don't compare Floats for equality. Compare for less than or greater than, but never exact equality.

http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding

Simply, it is not the case that "it should be exact". Don't expect that from floating point decimals.

Joe Koberg
  • 22,019
  • 6
  • 45
  • 52
2

Welcome to floating point math. There are many numbers which cannot be represented in standard floating point notation and come out just a tiny bit off.

This is easily illustrated as follows:

(1..10).collect do |i|
  v = ((10**i).to_f + 0.7)
  puts "%13.1f = %.30f" % [ v, v.modulo(1) ]
end

Where the result is:

         10.7 = 0.699999999999999289457264239900
        100.7 = 0.700000000000002842170943040401
       1000.7 = 0.700000000000045474735088646412
      10000.7 = 0.700000000000727595761418342590
     100000.7 = 0.699999999997089616954326629639
    1000000.7 = 0.699999999953433871269226074219
   10000000.7 = 0.699999999254941940307617187500
  100000000.7 = 0.700000002980232238769531250000
 1000000000.7 = 0.700000047683715820312500000000
10000000000.7 = 0.700000762939453125000000000000

Notice that the larger the number gets, the lower the precision beyond the decimal place. This is because there is a fixed amount of precision available to represent the entire number.

tadman
  • 194,930
  • 21
  • 217
  • 240
1

Floating points are not exact. The short version is it's not possible to store an infinite amount of values in a finite amount of bits.

The longer version is What Every Computer Scientist Should Know About Floating-Point Arithmetic

leeeroy
  • 10,518
  • 17
  • 48
  • 52
0

this is because it's not possible to represent all floating point numbers exactly.

Peter
  • 114,224
  • 47
  • 167
  • 205