8

64bit file API is different on each platform.

in windows: _fseeki64
in linux: fseeko
in freebsd: yet another similar call ...

How can I most effectively make it more convenient and portable? Are there any useful examples?

jcolebrand
  • 15,923
  • 10
  • 71
  • 117
Mad Fish
  • 693
  • 1
  • 8
  • 12

5 Answers5

14

Most POSIX-based platforms support the "_FILE_OFFSET_BITS" preprocessor symbol. Setting it to 64 will cause the off_t type to be 64 bits instead of 32, and file manipulation functions like lseek() will automatically support the 64 bit offset through some preprocessor magic. From a compile-time point of view adding 64 bit file offset support in this manner is fairly transparent, assuming you are correctly using the relevant typedefs. Naturally, your ABI will change if you're exposing interfaces that use the off_t type. Ideally you should define it on the command line, e.g.:

cxx -D_FILE_OFFSET_BITS=64

to make sure it applies to all OS headers included by your code.

Unfortunately, Windows doesn't support this preprocessor symbol so you'll either have to handle it yourself, or rely on a library that provides cross-platform large file support. ACE is one such library (both POSIX based and Windows platforms - just define _FILE_OFFSET_BITS=64 in both cases). I know that Boost.filesystem also supports large files on POSIX based platforms, but I'm not sure about Windows. Other cross-platform libraries likely provide similar support.

Void
  • 3,321
  • 16
  • 18
  • how to set this flag from visual studios 2008? – savi Feb 23 '15 at 22:38
  • Can you help me get this in Visual studios? – savi Feb 23 '15 at 22:53
  • @savi: I'm not aware of a Visual Studio equivalent to `_FILE_OFFSET_BITS=64`. If you're not using a library that transparently handles the large file offsets for you, you'll likely have to roll your own code to achieve the same affect. For example, the discussion "[Huge file access in C](http://coding.derkeiler.com/Archive/C_CPP/comp.lang.c/2006-12/msg03560.html)" from the comp.lang.c newsgroup describes one approach. – Void Feb 24 '15 at 21:50
  • Is there an alternative that I can look at? I am using a 32bit Winodws7 machine. The example from comp.lang.c does not help either. – savi Feb 25 '15 at 19:00
  • You do mention about ACE does support Windows, how do I use ACE to solve my problem? – savi Feb 25 '15 at 20:20
2

My best suggestion would be to use a library for handling this which already exists.

A good candidate might be using the CPL file library from GDAL. This provides cross-platform large-file support for read+write, in ascii and binary access. Most of the GDAL file formats have been implemented using it, and are widely used.

Reed Copsey
  • 522,342
  • 70
  • 1,092
  • 1,340
0

STLport have native support for 64bits file size. On the other hand if you're doing intensive disk accesses, you might want to use file mapping : mmap on unix and CreateFileMapping on Windows.

diapir
  • 2,493
  • 1
  • 15
  • 24
0

I wrote a library once called 'mfile', which stands for 'multi-file'. The problem was that I had a system that didn't support large files, so this library would either use the largefile support available on some systems, or handle being passed multiple files that it would virtually concat.

At any rate, it provided a 'FILE'-like interface, and worked fairly well. When I had to port to Windows, it was easy to put in the appropriate 64 bit support. On Unix, _FILE_OFFSET_BITS provide enough magic that I didn't have to play too many games for it to work.

Chris Arguin
  • 11,370
  • 4
  • 30
  • 46
0

Write a class that encapsulates file-handling, and use conditional compilation in the class to handle each platform.

Then just use the class instead of the native file-handling when you need to access a file from your program.

As far as I can tell, there's no set of universally portable file I/O functions when it comes large file handling.

Marc Bernier
  • 2,844
  • 23
  • 43