Questions tagged [floating-point]

Floating point numbers are approximations of real numbers that can represent larger ranges than integers but use the same amount of memory, at the cost of lower precision. If your question is about small arithmetic errors (e.g. why does 0.2 + 0.1 equal 0.300000001?) or decimal conversion errors, please read the "info" page linked below before posting.

Many questions asked here about floating point math are about small inaccuracies in floating point arithmetic. To use the example from the excerpt, 0.1 + 0.1 + 0.1 might result in 0.300000001 instead of the expected 0.3. Errors like these are caused by the way floating point numbers are represented in computers' memory.

Integers are stored as exact values of the numbers they represent. Floating point numbers are stored as two values: a significand and an exponent. It is not possible to find a significand-exponent pair that matches every possible real number. As a result, some approximation and therefore inaccuracy is unavoidable.

Two commonly cited introductory-level resources about floating point math are What Every Computer Scientist Should Know About Floating-Point Arithmetic and the floating-point-gui.de.

FAQs:

Why 0.1 does not exist in floating point

Floating Point Math at https://0.30000000000000004.com/

Related tags:

Programming languages where all numbers are double-precision (64b) floats:

13427 questions
579
votes
5 answers

Why does Math.round(0.49999999999999994) return 1?

In the following program you can see that each value slightly less than .5 is rounded down, except for 0.5. for (int i = 10; i >= 0; i--) { long l = Double.doubleToLongBits(i + 0.5); double x; do { x =…
Peter Lawrey
  • 498,481
  • 72
  • 700
  • 1,075
577
votes
32 answers

What is the most effective way for float and double comparison?

What would be the most efficient way to compare two double or two float values? Simply doing this is not correct: bool CompareDoubles1 (double A, double B) { return A == B; } But something like: bool CompareDoubles2 (double A, double B) { …
Alex
  • 5,795
  • 3
  • 15
  • 3
542
votes
27 answers

How to nicely format floating numbers to string without unnecessary decimal 0's

A 64-bit double can represent integer +/- 253 exactly. Given this fact, I choose to use a double type as a single type for all my types, since my largest integer is an unsigned 32-bit number. But now I have to print these pseudo integers, but the…
Pyrolistical
  • 26,088
  • 21
  • 78
  • 104
523
votes
5 answers

Correct format specifier for double in printf

What is the correct format specifier for double in printf? Is it %f or is it %lf? I believe it's %f, but I am not sure. Code sample #include int main() { double d = 1.4; printf("%lf", d); // Is this wrong? }
Leopard
  • 5,241
  • 3
  • 13
  • 4
508
votes
5 answers

How to get a random number between a float range?

randrange(start, stop) only takes integer arguments. So how would I get a random number between two float values?
Mantis Toboggan
  • 5,645
  • 3
  • 16
  • 10
458
votes
17 answers

How to parse float with two decimal places in javascript?

I have the following code. I would like to have it such that if price_result equals an integer, let's say 10, then I would like to add two decimal places. So 10 would be 10.00. Or if it equals 10.6 would be 10.60. Not sure how to do…
user357034
  • 9,761
  • 18
  • 53
  • 70
456
votes
13 answers

What is the difference between float and double?

I've read about the difference between double precision and single precision. However, in most cases, float and double seem to be interchangeable, i.e. using one or the other does not seem to affect the results. Is this really the case? When are…
VaioIsBorn
  • 6,893
  • 9
  • 29
  • 27
450
votes
14 answers

How to format a float in javascript?

In JavaScript, when converting from a float to a string, how can I get just 2 digits after the decimal point? For example, 0.34 instead of 0.3445434.
F40
404
votes
11 answers

How dangerous is it to compare floating point values?

I know UIKit uses CGFloat because of the resolution independent coordinate system. But every time I want to check if for example frame.origin.x is 0 it makes me feel sick: if (theView.frame.origin.x == 0) { // do important operation } Isn't…
Proud Member
  • 38,700
  • 43
  • 143
  • 225
387
votes
16 answers

What is the best way to compare floats for almost-equality in Python?

It's well known that comparing floats for equality is a little fiddly due to rounding and precision issues. For example: https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/ What is the recommended way to deal…
Gordon Wrigley
  • 9,129
  • 8
  • 41
  • 59
374
votes
16 answers

How do I print a double value with full precision using cout?

In my earlier question I was printing a double using cout that got rounded when I wasn't expecting it. How can I make cout print a double using full precision?
Jason Punyon
  • 37,168
  • 13
  • 93
  • 118
371
votes
8 answers

How to convert float to int with Java

I used the following line to convert float to int, but it's not as accurate as I'd like: float a=8.61f; int b; b=(int)a; The result is : 8 (It should be 9) When a = -7.65f, the result is : -7 (It should be -8) What's the best way to do it ?
Frank
  • 28,342
  • 54
  • 158
  • 227
370
votes
11 answers

JavaScript displaying a float to 2 decimal places

I wanted to display a number to 2 decimal places. I thought I could use toPrecision(2) in JavaScript . However, if the number is 0.05, I get 0.0500. I'd rather it stay the same. See it on JSbin. What is the best way to do this? I can think of coding…
alex
  • 438,662
  • 188
  • 837
  • 957
343
votes
6 answers

Double vs. BigDecimal?

I have to calculate some floating point variables and my colleague suggest me to use BigDecimal instead of double since it will be more precise. But I want to know what it is and how to make most out of BigDecimal?
Truong Ha
  • 9,070
  • 10
  • 35
  • 45
311
votes
2 answers

What does the constant 0.0039215689 represent?

I keep seeing this constant pop up in various graphics header files 0.0039215689 It seems to have something to do with color maybe? Here is the first hit on Google: void RDP_G_SETFOGCOLOR(void) { Gfx.FogColor.R = _SHIFTR(w1, 24, 8) *…
crush
  • 15,889
  • 8
  • 54
  • 95