69

I'm working on an implementation of the memcache protocol which, at some points, uses 64 bits integer values. These values must be stored in "network byte order".

I wish there was some uint64_t htonll(uint64_t value) function to do the change, but unfortunately, if it exist, I couldn't find it.

So I have 1 or 2 questions:

  • Is there any portable (Windows, Linux, AIX) standard function to do this ?
  • If there is no such function, how would you implement it ?

I have in mind a basic implementation but I don't know how to check the endianness at compile-time to make the code portable. So your help is more than welcome here ;)

Thank you.


Here is the final solution I wrote, thanks to Brian's solution.

uint64_t htonll(uint64_t value)
{
    // The answer is 42
    static const int num = 42;

    // Check the endianness
    if (*reinterpret_cast<const char*>(&num) == num)
    {
        const uint32_t high_part = htonl(static_cast<uint32_t>(value >> 32));
        const uint32_t low_part = htonl(static_cast<uint32_t>(value & 0xFFFFFFFFLL));

        return (static_cast<uint64_t>(low_part) << 32) | high_part;
    } else
    {
        return value;
    }
}
ereOn
  • 48,328
  • 33
  • 147
  • 228
  • 2
    possible duplicate of this one http://stackoverflow.com/questions/809902/64-bit-ntohl-in-c – INS Jun 11 '10 at 13:05
  • @ereOn: I also have similar question [here](http://stackoverflow.com/questions/19393539/how-to-swap-64-bit-integer-while-extracting-bytes-from-bytearray-in-c). If possible can you take a look and let me know what wrong I am doing here? – AKIWEB Oct 16 '13 at 05:10
  • 1
    Instead of including your answer inside the question, you should let your answer with answers. It is more readable. – mpromonet Aug 29 '15 at 15:24

7 Answers7

20
#define htonll(x) ((1==htonl(1)) ? (x) : ((uint64_t)htonl((x) & 0xFFFFFFFF) << 32) | htonl((x) >> 32))
#define ntohll(x) ((1==ntohl(1)) ? (x) : ((uint64_t)ntohl((x) & 0xFFFFFFFF) << 32) | ntohl((x) >> 32))

The test (1==htonl(1)) simply determines (at runtime sadly) if the hardware architecture requires byte swapping. There aren't any portable ways to determine at compile-time what the architecture is, so we resort to using "htonl", which is as portable as it gets in this situation. If byte-swapping is required, then we swap 32 bits at a time using htonl (remembering to swap the two 32 bit words as well).


Here's another way to perform the swap that is portable across most compilers and operating systems, including AIX, BSDs, Linux, and Solaris.

#if __BIG_ENDIAN__
# define htonll(x) (x)
# define ntohll(x) (x)
#else
# define htonll(x) ((uint64_t)htonl((x) & 0xFFFFFFFF) << 32) | htonl((x) >> 32))
# define ntohll(x) ((uint64_t)ntohl((x) & 0xFFFFFFFF) << 32) | ntohl((x) >> 32))
#endif

The important part is to use __BIG_ENDIAN__ or __LITTLE_ENDIAN__; and not __BYTE_ORDER__, __ORDER_BIG_ENDIAN__ or __ORDER_LITTLE_ENDIAN__. Some compilers and operating systems lack __BYTE_ORDER__ and friends.

TreyA
  • 3,129
  • 2
  • 15
  • 24
deltamind106
  • 403
  • 3
  • 11
  • 1
    I'm sure this answer is great, but could you please add something other than code to explain what and why? – AndyG Feb 18 '15 at 20:40
  • Thanks! I've edited your answer with the explanation. – AndyG Feb 19 '15 at 14:55
  • as mentioned @ereon you cannot use 32 bits shifts on `ntohl` or `ntohl` because they return `uint32_t`, hence triggering UB. – Aif May 17 '17 at 11:08
18

You are probably looking for bswap_64 I think it is supported pretty much everywhere but I wouldn't call it standard.

You can easily check the endianness by creating an int with a value of 1, casting your int's address as a char* and checking the value of the first byte.

For example:

int num = 42;
if(*(char *)&num == 42)
{
   //Little Endian
}
else
{
   //Big Endian
} 

Knowing this you could also make a simple function that does the swapping.


You could also always use boost which contains endian macros which are portable cross platform.

Brian R. Bondy
  • 314,085
  • 114
  • 576
  • 619
  • Thanks ;) The `boost` macros seem interesting. Do you have link ? – ereOn Jun 11 '10 at 12:41
  • 4
    Technically, just a byte-swap is not always enough. There are systems that are middle-endian (though no *modern* systems are). Also, `bswap_64` is definitely *NOT* supported everywhere; just look at the long list of systems with `bswap_64` missing: https://www.gnu.org/software/gnulib/manual/html_node/bswap_005f64.html – Tim Čas Feb 11 '15 at 13:20
  • Note that `bswap_64` is not [POSIX compliant](http://stackoverflow.com/questions/1780599/i-never-really-understood-what-is-posix). – patryk.beza Nov 09 '16 at 23:14
  • `bswap_64` is not present on Solaris i86pc. Does anyone know if it is present on AIX? – jww May 04 '19 at 09:19
7

You can try with uint64_t htobe64(uint64_t host_64bits) & uint64_t be64toh(uint64_t big_endian_64bits) for vice-versa.

glennsl
  • 23,127
  • 11
  • 49
  • 65
suresh m
  • 545
  • 5
  • 13
5

This seems to work in C; did I do anything wrong?

uint64_t htonll(uint64_t value) {
    int num = 42;
    if (*(char *)&num == 42) {
        uint32_t high_part = htonl((uint32_t)(value >> 32));
        uint32_t low_part = htonl((uint32_t)(value & 0xFFFFFFFFLL));
        return (((uint64_t)low_part) << 32) | high_part;
    } else {
        return value;
    }
}
pix0r
  • 30,601
  • 18
  • 82
  • 102
1

To reduce the overhead of the "if num == ..." Use the pre-processor defines:

#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
#else
#endif
  • Not portable. `__BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__` is a Linux thing. AIX lacks it, and will drop into the big-endian code path. – jww May 04 '19 at 09:44
0

Well, I think it's better to use the endian switching at compile time as much as possible, but I prefer to use function instead of macro, because in a macro the parameters are just replaced by the arguments; so arguments can be evaluated multiple times and create weird result if they are present multiple times in the macro (as done in some of the previous provided solutions).

uint64_t htonll(uint64_t x)
{
#if __BIG_ENDIAN__
    return x;
#else
    return ((uint64_t)htonl((x) & 0xFFFFFFFFLL) << 32) | htonl((x) >> 32);
#endif
}

uint64_t ntohll(uint64_t x)
{
#if __BIG_ENDIAN__
    return x;
#else
    return ((uint64_t)ntohl((x) & 0xFFFFFFFFLL) << 32) | ntohl((x) >> 32);
#endif
}

So this allow calling htonll(x++) without increment x several time like it will be done using the previous macros.

-3

EDIT: combining the two (used Brian's code):

uint64_t htonll(uint64_t value)
{
     int num = 42;
     if(*(char *)&num == 42)
          return (htonl(value & 0xFFFFFFFF) << 32LL) | htonl(value >> 32);
     else 
          return value;
}

Warning: untested code! Please test before using.

Pavel Radzivilovsky
  • 17,994
  • 3
  • 54
  • 65
  • @Pavel Still doesn't work. `htonl()` returns a 32 bits value on which you **cannot** call `<< 32LL` (because you cannot shift 32 bits left on an only 32 bits value). – ereOn Jun 11 '10 at 13:08
  • I think shifting by 32LL will do promotion of the left side, no? – Pavel Radzivilovsky Jun 11 '10 at 15:03
  • @Pavel: no, the bit-size of the shift value doesn't change anything. gcc emits a warning: "left shift count >= width of type". And your function says that htonll(0x0102030405060708ULL) == 0xc070605. – bstpierre Aug 02 '10 at 19:52
  • Change it to `return ((uint64_t)htonl(value & 0xFFFFFFFF) << 32LL) | htonl(value >> 32);` and it works right. – bstpierre Aug 02 '10 at 19:55