37

For floating point values, is it guaranteed that a + b == b + a ?

I believe this is guaranteed in IEEE754, however the C++ standard does not specify that IEEE754 must be used. The only relevant text seems to be from [expr.add]#3:

The result of the binary + operator is the sum of the operands.

The mathematical operation "sum" is commutative. However, the mathematical operation "sum" is also associative, whereas floating point addition is definitely not associative. So, it seems to me that we cannot conclude that the commutativity of "sum" in mathematics means that this quote specifies commutativity in C++.

M.M
  • 130,300
  • 18
  • 171
  • 314
  • 4
    `std::strtod("nan")+0.0 == 0.0+std::strtod("nan")` is false. But I doubt that's what you mean. – aschepler Jun 27 '14 at 01:42
  • 2
    Why limit yourself to floating-point? Without giving it too much thought, I don’t think that C++ requires integer addition be commutative either (e.g. overflow results could be non-commutative). – Stephen Canon Jun 27 '14 at 01:55
  • 3
    @StephenCanon: That would seem to depend on what treatment you feel like giving undefined behaviour. Integer addition is commutative for programs that don't elicit UB, and whether a program elicits UB cannot hinge on commutativity of integer addition. – tmyklebu Jun 27 '14 at 02:06

3 Answers3

19

No, the C++ language generally wouldn't make such a requirement of the hardware. Only the associativity of operators is defined.

All kinds of crazy things do happen in floating-point arithmetic. Perhaps, on some machine, adding zero to an denormal number produces zero. Conceivable that a machine could avoid updating memory in the case of adding a zero-valued register to a denormal in memory. Possible that a really dumb compiler would always put the LHS in memory and the RHS in a register.

Note, though, that a machine with non-commutative addition would need to specifically define how expressions map to instructions, if you're going to have any control over which operation you get. Does the left-hand side go into the first machine operand or the second?

Such an ABI specification, mentioning the construction of expressions and instructions in the same breath, would be quite pathological.

Potatoswatter
  • 126,977
  • 21
  • 238
  • 404
  • 1
    "Only the associativity is fixed." what do you mean by 'fixed'? – huon Jun 27 '14 at 01:44
  • 1
    @dbaupp "Defined." Arithmetic operations are left-associative. Updated. – Potatoswatter Jun 27 '14 at 01:45
  • 12
    Operator associativity is different from the mathematical property of associativity of multiplication. The former says that `double x = a * b * c;` will be evaluated in a left-associative manner as `(a * b) * c`. In math, associativity means that `(a * b) * c == a * (b * c)`. Floating point addition/multipliation is evaluated left-associative, but is definitely not mathematically associative (i.e. a right-associative evaluation could give a different result). – TemplateRex Jun 27 '14 at 22:04
  • @TemplateRex Hmm, I don't see how that relates to the question or my answer. This discussion is entirely about notation used for programming floating-point calculations, not the mathematical operations which are being approximated. – Potatoswatter Jun 28 '14 at 16:18
  • 5
    I simply pointed out that associativity has two meanings and it wasn't clear from the context which one you were referring to. – TemplateRex Jun 28 '14 at 16:32
18

It is not even required that a + b == a + b. One of the subexpressions may hold the result of the addition with more precision than the other one, for example when the use of multiple additions requires one of the subexpressions to be temporarily stored in memory, when the other subexpression can be kept in a register (with higher precision).

If a + b == a + b is not guaranteed, a + b == b + a cannot be guaranteed. If a + b does not have to return the same value each time, and the values are different, one of them necessarily will not be equal to one particular evaluation of b + a.

  • You seem to be saying that `a + b` is unspecified then. Can you elaborate on this? It's normally stated that the reason the compiler is not allowed to optimize floating point operations is that it might change the observable behaviour of the program. However if the result of `a + b` can change between invocations , that would invalidate that line of reasoning. – M.M Jun 30 '14 at 23:19
  • 1
    @MattMcNabb It depends on which rules the compiler wants to follow. The C and C++ standards are very lenient, but because they are so lenient, an implementation may implement stricter rules (possibly conforming to other standards) and still conform to the C++ standard as well. Such stricter rules would indeed disallow many floating point optimisations. The C++ standard states in [expr.prim.general]p11: "The values of the floating operands and the results of floating expressions may be represented in greater precision and range than that required by the type; the types are not changed thereby." –  Jun 30 '14 at 23:50
  • 1
    @MattMcNabb What this means in practise is that, as an example, if `a` and `b` are of type `double`, and `a + b` is exactly representable in `long double` yet would require rounding in `double`, and the CPU only has a single floating-point type with the precision of `long double`, then you might get the result of `(long double) a + (long double) b` instead. Both the C and the C++ standards intentionally do not want to require the compiler to emit a round instruction after each and every single floating-point operation. –  Jun 30 '14 at 23:56
  • [conv.double] says that if a `long double` value is not exactly representable in `double` then the result is implementation-defined. (So it should give the same value each time). Does [conv.double] apply here? – M.M Jul 01 '14 at 00:09
  • 1
    @MattMcNabb That doesn't apply: there isn't any `long double` value. There is a `double` value with the precision of `long double`, and no actual conversion between different types. And implementation-defined doesn't mean it has to give the same value each time, it merely means the implementation has to document how it chooses the result: implementations may provide options to change the rounding mode at run time. –  Jul 01 '14 at 05:04
  • 1
    OK. Has there been discussion of this anywhere else (i.e. whether `a + b == a + b` can be false in C++)? Not that I distrust you or anything; but when the standard avoids specifically covering an issue, it can be insightful to see some discussion between opposing viewpoints. – M.M Jul 01 '14 at 05:13
  • 4
    There's GCC's [bug 323](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=323), about a program that does assume that `a + b == a + b`, where the GCC developers have closed it as not a bug. –  Jul 01 '14 at 05:41
  • interesting point on that thread that this implies `std::set` is broken – M.M Jul 01 '14 at 05:47
  • 1
    Correction: its status is actually suspended, not resolved. Regardless, it's a good read. –  Jul 01 '14 at 05:47
  • 2
    Worse than the fact that `a+b == a+b` isn't guaranteed, IEEE-754 requires that some values of `a` won't even satisfy `a==a`. – supercat May 04 '15 at 20:55
  • @supercat do you mean `NaN` and `Inf` with 'some values'? Or does it also hold for "real" numbers – Kami Kaze Dec 20 '16 at 08:49
  • 2
    @KamiKaze: I was referring to NaN. While it's possible to write a NaN-safe floating-point sort that behaves sensibly by using an initial pass to partition the data into NaN and everything else, and then sorting the part that doesn't contain any NaN values, that's an extra layer of complexity that could have been avoided by defining a relational operator that works consistently. – supercat Dec 23 '16 at 00:02
  • @supercat I would say a false evaluation of NaN is quite sensible, because it arises under circumstances that really are an "error" and they should not be compared. – Kami Kaze Dec 23 '16 at 08:36
  • 1
    @KamiKaze: If one is trying to e.g. identify all the unique values in a list, cache a pure function's inputs and outputs, compare old and new copies of a list, etc., having NaN compare unequal to itself is decidedly unhelpful. Equivalence relations and pure functions are useful concepts--if f() is a pure function, and code knows that that x==y, and f(x) is z, code that needs f(y) can substitute f(x). If == operates as an equivalence relation, code can handle NaN like any other value. Otherwise special handling will be needed to avoid having every request for f(NaN) add a new entry.. – supercat Dec 23 '16 at 23:24
  • 1
    ...to the table (which will never match anything and may as well not exist). – supercat Dec 23 '16 at 23:25
11

The C++ standard very specifically does not guarantee IEEE 754. The library does have some support for IEC 559 (which is basically just the IEC's version of the IEEE 754 standard), so you can check whether the underlying implementation uses IEEE 754/IEC 559 though (and when it does, you can depend on what it guarantees, of course).

For the most part, the C and C++ standards assume that such basic operations will be implemented however the underlying hardware works. For something as common as IEEE 754, they'll let you detect whether it's present, but still don't require it.

Jerry Coffin
  • 437,173
  • 71
  • 570
  • 1,035
  • Howdy! I +1'd you. Say hi in Chat for me; my ISP has been having border gateway protocol issues (or something) and chat is unreachable from here :( . I wonder, has Mark Garcia been around? He might be on the same ISP. – Potatoswatter Jun 27 '14 at 10:32
  • @Potatoswatter: I'll say hi for you. Don't recall seeing Mark around much lately either, now that you mention it. – Jerry Coffin Jun 27 '14 at 15:41
  • I've accepted hvd's answer (for now at least) - assuming it to be correct, it indicates that even on IEEE754 implementations , if the result is not exactly representable then we do not even have identity, let alone commutativity. – M.M Jul 01 '14 at 05:50