Are floating point operations in C associative?

Question

Addition mathematically holds the associative property:

(a + b) + c = a + (b + c)

In the general case, this property does not hold for floating-point numbers because they represent values in a finite precision.

Is a compiler allowed to make the above substitution when generating machine code from a C program as part of an optimization? Where does it exactly say in the C standard?

For multiplication at least, look here: http://stackoverflow.com/questions/6430448/why-doesnt-gcc-optimize-aaaaaa-to-aaaaaa — ta.speot.is, Dec 17 '12 at 11:15

score 15 · Accepted Answer · answered Dec 17 '12 at 11:32

The compiler is not allowed to perform "optimizations", which would result in a different value computed, than the one computed according to abstract machine semantics.

5.1.2.3 Program execution

[#1] The semantic descriptions in this International Standard describe the behavior of an abstract machine in which issues of optimization are irrelevant.

[#3] In the abstract machine, all expressions are evaluated as specified by the semantics.

[#13] EXAMPLE 5 Rearrangement for floating-point expressions is often restricted because of limitations in precision as well as range. The implementation cannot generally apply the mathematical associative rules for addition or multiplication, nor the distributive rule, because of roundoff error, even in the absence of overflow and underflow.

In your example:

(a + b) + c

or even without the parentheses:

a + b + c

we have

   +
  / \
  +  c
 / \
 a  b

and the compiler is required to generate code as if a is summed with b and the result is summed with c.

Correct theory. The application to take home is: work out what order your operations should happen in (eg if you want to sum small to small, big to big), put in parentheses to satisfy paranoia, and you can stop worrying about the compiler breaking things by reordering. — Nicholas Wilson, Dec 17 '12 at 11:37
Note however, that while operator precedence is well-defined, the order of evaluation of the sub expressions is unspecified. In other words, the program may evaluate this binary tree starting right-to-left, or left-to-right, and it doesn't even have to keep to this order in a consistent manner, nor does it need to document it. So if either a, b or c contained side-effects that affected the result, then the code would be problematic. Say for example that c is a function that modifies a: then we couldn't know the result. — Lundin, Dec 17 '12 at 11:56

score 7 · Answer 2 · edited Mar 25 '19 at 20:41

You can make floating point operations associative with the gcc options:

-funsafe-math-optimizations -O2

Example:

double test (double a, double b, double c) {    
  return (a + b + c) * (a + (b + c));
}

This is reduced to:

double temp = a + (b + c);
return temp * temp;

Similarly, (a + b + c) - (a + (b + c)) is reduced to zero, ignoring the possibility of INF and NAN.

If I compile with -fassociative-math -O2 instead, I get the weird message:

warning: -fassociative-math disabled; other options take precedence

The -funsafe-math-optimizations can improve speed if you don't care about the order of the operands anyway, but it may cause loss of precision if the order of operands is important, and you may lose NAN and INF results.

score 4 · Answer 3 · answered Jun 22 '14 at 20:32

4

Floating point multiplication in C is not associative.

In C, Floating point multiplication is not associative.

Some evidence is with this C code:

Pick three random float values.
Check if a*(b*c) is ever not equal to (a*b)*c

#include<stdio.h>
#include<time.h>
#include<stdlib.h>
using namespace std;
int main() {
    int counter = 0;
    srand(time(NULL));
    while(counter++ < 10){
        float a = rand() / 100000;
        float b = rand() / 100000;
        float c = rand() / 100000;

        if (a*(b*c) != (a*b)*c){
            printf("Not equal\n");
        }
    }
    printf("DONE");
    return 0;
}

The program prints:

Not equal
Not equal
Not equal
Not equal
DONE
RUN FINISHED; exit value 0; real time: 10ms; user: 0ms; system: 0ms

Conclusion:

For my test, three randomly selected floating point multiplication values are associative about 70% of the time.

answered Jun 22 '14 at 20:32

Eric Leschinski

123,728
82
382
321

1

The OP asked whether the compiler *assumes* float operations to be associative, when it performs optimizations. He is clearly aware that they actually aren't. – Jørgen Fogh Nov 10 '15 at 07:55
2

Yes but its always nice to see an example which confirms the theory. – Eric Leschinski Nov 10 '15 at 12:35
This isn't about c, however, it's just telling you something about float point math. Any execution environment that conforms to the IEEE standard will exhibit the same behvior. – dmckee --- ex-moderator kitten Sep 18 '19 at 22:24

Are floating point operations in C associative?

3 Answers3

Linked