Behavior of SO_REUSEADDR and SO_REUSEPORT changed?

Question

In older versions of Mac OS X, wildcard binding works like described here:

Socket options SO_REUSEADDR and SO_REUSEPORT, how do they differ? Do they mean the same across all major operating systems?

Multicast Addresses

The meaning of SO_REUSEADDR changes for multicast addresses as it allows multiple sockets to be bound to exactly the same combination of source multicast address and port. In other words, for multicast addresses SO_REUSEADDR behaves exactly as SO_REUSEPORT for unicast addresses. Actually the code treats SO_REUSEADDR and SO_REUSEPORT identically for multicast addresses, that means you could say that SO_REUSEADDR implies SO_REUSEPORT for all multicast addresses and the other way round.

and

MacOS X

At its very core, MacOS X is simply a BSD-style UNIX, based on a rather late fork of the BSD code, which was even synchronized with FreeBSD 5 for the Mac OS 10.3 release. That's why MacOS X offers the same options as BSD and they also behave the same way as in BSD.

but in 10.10.5 I discovered a change when I was testing my networking library.

Two unbound (wildcard) UDP sockets can not share the same port anymore (errno=EADDRINUSE), even when SO_REUSEADDR is set. SO_REUSEPORT has to be set on both two, which is a mystery to me.

It is reproducible with this simple test code:

#include <arpa/inet.h>
#include <sys/socket.h>
#include <sys/ioctl.h>
#include <unistd.h>
#include <string.h>
#include <stdio.h>
#include <errno.h>

int main() {
    for(int p = 0; p < 4; ++p) {
        printf("Flags set: ");
        if(p&1) printf("SO_REUSEADDR ");
        if(p&2) printf("SO_REUSEPORT");
        printf("\n");

        int handles[2];
        bool success = true;
        for(int i = 0; i < sizeof(handles)/sizeof(int); ++i) {
            handles[i] = socket(AF_INET, SOCK_DGRAM, 0);

            int flag = 1;
            if((p&1) && setsockopt(handles[i], SOL_SOCKET, SO_REUSEADDR, &flag, sizeof(flag)) == -1) {
                printf("Setsockopt %d, SO_REUSEADDR failed with errno\n", i, errno);
                success = false;
                break;
            }
            if((p&2) && setsockopt(handles[i], SOL_SOCKET, SO_REUSEPORT, &flag, sizeof(flag)) == -1) {
                printf("Setsockopt %d, SO_REUSEPORT failed with errno\n", i, errno);
                success = false;
                break;
            }

            struct sockaddr_in addr;
            memset(&addr, 0, sizeof(addr));
            addr.sin_family = AF_INET;
            addr.sin_port = 2000; // does not matter as long as it is currently free
            addr.sin_addr.s_addr = 0; // wildcard address

            if(bind(handles[i], (struct sockaddr*)&addr, sizeof(addr)) == -1) {
                printf("Bind %d failed with errno %d\n", i, errno);
                success = false;
                break;
            }
        }
        if(success)
            printf("Alright\n");

        for(int i = 0; i < sizeof(handles)/sizeof(int); ++i)
            close(handles[i]);
        printf("\n");
    }

    return 0;
}

Which outputs:

Flags set: 
Bind 1 failed with errno 48

Flags set: SO_REUSEADDR 
Bind 1 failed with errno 48

Flags set: SO_REUSEPORT
Alright

Flags set: SO_REUSEADDR SO_REUSEPORT
Alright

Answer 1

In older versions of Mac OS X, wildcard binding works like described here:

Socket options SO_REUSEADDR and SO_REUSEPORT, how do they differ? Do they mean the same across all major operating systems?

Multicast Addresses
…

The description you quote is for Multicast Addresses. Your test code doesn't use a multicast address. Hence, a different description applies (from the same source):

SO_REUSEPORT

SO_REUSEPORT is what most people would expect SO_REUSEADDR to be. Basically, SO_REUSEPORT allows you to bind an arbitrary number of sockets to exactly the same source address and port as long as all prior bound sockets also had SO_REUSEPORT set before they were bound. …

Your test code confirms exactly this.

Answer 2

What a hack! It just has bee incurred. Still the problem exists with using SO_REUSEADDR instead of SO_REUSEPORT. I think the latter works most of *nix based OSes. But once you get caught into the nest, shaking off the situation would not be easy, if you didn't read the answer before especially on Mac OS.

You know your port number of course, while using sockets. Open terminal, execute the following command.

lsof -i:<Port used by you>

then it brings you a line including PID. Just kill it with -9, then change your code by SO_REUSEPORT. That's all!