Questions tagged [floating-point]

Floating point numbers are approximations of real numbers that can represent larger ranges than integers but use the same amount of memory, at the cost of lower precision. If your question is about small arithmetic errors (e.g. why does 0.2 + 0.1 equal 0.300000001?) or decimal conversion errors, please read the "info" page linked below before posting.

Many questions asked here about floating point math are about small inaccuracies in floating point arithmetic. To use the example from the excerpt, 0.1 + 0.1 + 0.1 might result in 0.300000001 instead of the expected 0.3. Errors like these are caused by the way floating point numbers are represented in computers' memory.

Integers are stored as exact values of the numbers they represent. Floating point numbers are stored as two values: a significand and an exponent. It is not possible to find a significand-exponent pair that matches every possible real number. As a result, some approximation and therefore inaccuracy is unavoidable.

Two commonly cited introductory-level resources about floating point math are What Every Computer Scientist Should Know About Floating-Point Arithmetic and the floating-point-gui.de.

FAQs:

Why 0.1 does not exist in floating point

Floating Point Math at https://0.30000000000000004.com/

Related tags:

Programming languages where all numbers are double-precision (64b) floats:

13427 questions
301
votes
3 answers

Why does NaN - NaN == 0.0 with the Intel C++ Compiler?

It is well-known that NaNs propagate in arithmetic, but I couldn't find any demonstrations, so I wrote a small test: #include #include int main(int argc, char* argv[]) { float qNaN = std::numeric_limits::quiet_NaN(); …
imallett
  • 13,288
  • 9
  • 49
  • 115
301
votes
6 answers

Why does changing the sum order returns a different result?

Why does changing the sum order returns a different result? 23.53 + 5.88 + 17.64 = 47.05 23.53 + 17.64 + 5.88 = 47.050000000000004 Both Java and JavaScript return the same results. I understand that, due to the way floating point numbers are…
Marlon Bernardes
  • 11,021
  • 6
  • 34
  • 43
300
votes
13 answers

Make a float only show two decimal places

I have the value 25.00 in a float, but when I print it on screen it is 25.0000000. How can I display the value with only two decimal places?
ron
300
votes
20 answers

Why can't decimal numbers be represented exactly in binary?

There have been several questions posted to SO about floating-point representation. For example, the decimal number 0.1 doesn't have an exact binary representation, so it's dangerous to use the == operator to compare it to another floating-point…
Barry Brown
  • 19,087
  • 14
  • 65
  • 102
296
votes
9 answers

What MySQL data type should be used for Latitude/Longitude with 8 decimal places?

I'm working with map data, and the Latitude/Longitude extends to 8 decimal places. For example: Latitude 40.71727401 Longitude -74.00898606 I saw in the Google document which uses: lat FLOAT( 10, 6 ) NOT NULL, lng FLOAT( 10, 6 ) NOT…
Edward
  • 8,322
  • 16
  • 45
  • 66
294
votes
11 answers

What is the rationale for all comparisons returning false for IEEE754 NaN values?

Why do comparisons of NaN values behave differently from all other values? That is, all comparisons with the operators ==, <=, >=, <, > where one or both values is NaN returns false, contrary to the behaviour of all other values. I suppose this…
starblue
  • 51,675
  • 14
  • 88
  • 146
292
votes
3 answers

Float vs Decimal in ActiveRecord

Sometimes, Activerecord data types confuse me. Err, often. One of my eternal questions is, for a given case, Should I use :decimal or :float? I've often come across this link, ActiveRecord: :decimal vs :float?, but the answers aren't quite clear…
289
votes
14 answers

Random float number generation

How do I generate random floats in C++? I thought I could take the integer rand and divide it by something, would that be adequate enough?
hasen
  • 148,751
  • 62
  • 182
  • 223
288
votes
2 answers

Why are some float < integer comparisons four times slower than others?

When comparing floats to integers, some pairs of values take much longer to be evaluated than other values of a similar magnitude. For example: >>> import timeit >>> timeit.timeit("562949953420000.7 < 562949953421000") # run 1 million…
Alex Riley
  • 132,653
  • 39
  • 224
  • 205
285
votes
25 answers

Get decimal portion of a number with JavaScript

I have float numbers like 3.2 and 1.6. I need to separate the number into the integer and decimal part. For example, a value of 3.2 would be split into two numbers, i.e. 3 and 0.2 Getting the integer portion is easy: n = Math.floor(n); But I am…
Oscar
  • 2,861
  • 2
  • 13
  • 5
283
votes
6 answers

Why are these numbers not equal?

The following code is obviously wrong. What's the problem? i <- 0.1 i <- i + 0.05 i ## [1] 0.15 if(i==0.15) cat("i equals 0.15") else cat("i does not equal 0.15") ## i does not equal 0.15
dplanet
  • 4,883
  • 9
  • 26
  • 42
283
votes
9 answers

How to convert string into float in JavaScript?

I am trying to parse two values from a datagrid. The fields are numeric, and when they have a comma (ex. 554,20), I can't get the numbers after the comma. I've tried parseInt and parseFloat. How can I do this?
F40
276
votes
12 answers

When should I use double instead of decimal?

I can name three advantages to using double (or float) instead of decimal: Uses less memory. Faster because floating point math operations are natively supported by processors. Can represent a larger range of numbers. But these advantages seem to…
Jamie Ide
  • 45,803
  • 16
  • 74
  • 115
272
votes
7 answers

Why is division in Ruby returning an integer instead of decimal value?

For example: 9 / 5 #=> 1 but I expected 1.8. How can I get the correct decimal (non-integer) result? Why is it returning 1 at all?
ErwinM
  • 4,671
  • 2
  • 20
  • 32
272
votes
8 answers

Convert floats to ints in Pandas?

I've been working with data imported from a CSV. Pandas changed some columns to float, so now the numbers in these columns get displayed as floating points! However, I need them to be displayed as integers, or, without comma. Is there a way to…
MJP
  • 3,827
  • 5
  • 14
  • 16