How can I seekg() files over 4GB on Windows?

Question

I am on Windows 7, 64bit and NTFS. I am building a DLL that must be 32 bit. I have a very simple routine that I'd like to implement in C++. I'm reading a large file using:

unsigned long p;
ifstream source(file);
streampos pp(p);
source.seekg(pp);

For files over 4GB I tried using unsigned long long but it's not working. What am I doing wrong? I'm using GNU GCC, would it be of any use trying MSVC Express 2008/2010?

Update:

There seems that something is wrong with my GCC. Right now I'm testing your proposals using MSVC and it seems that is working. MSVC uses an _int64 to represent streampos/streamoff objects, I will check later with GCC.

I ran into this: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=8610 — Josep Valls, Feb 23 '12 at 14:39

score 2 · Answer 1 · answered Feb 23 '12 at 01:21

If you are running on a 32 bit system you are probably out of luck doing it the simple way although the streams library is free to use a 64 bit word for its pos_type. However, it might work to use relative seeks. Since all seeks return a pos_type which supposedly indicates the current position, this still might not work too well.

I guess it is just me but I never found seeking to be too useful anyway. Of course, having implemented this mess I'm also aware that seeking is bound to kill performance and that it only really works when using files opened in std::ios_base::binary mode which use no code conversion.

The seek operation is actually to retrieve very small information from a very large file, so seek should make sense. The file is indeed opened as binary. — Josep Valls, Feb 23 '12 at 15:00

score 1 · Accepted Answer · answered Feb 23 '12 at 01:17

You may have to use a number of relative seeks instead, i.e. use the two-argument overload of seekg.

// Start with seeking from the beginning
source.seekg(some_pos, std::ios::beg);

// Then seek some more from that position
source.seekg(some_offset, std::ios::cur);

score 1 · Answer 3 · answered Feb 23 '12 at 01:20

1

I believe you'll have to use native Win32 calls to do this like SetFilePointerEx http://msdn.microsoft.com/en-us/library/aa365542(VS.85).aspx

answered Feb 23 '12 at 01:20

Steven Behnke

3,190
2
24
33

That was my first thought, but I was trying to avoid that. I am building a library that I expect to be cross-platform. Eventually I want the library to be used in Python, which is itself cross-platform and supports +4GB file access. In the docs, they recommend building with MSVC for Windows deployments which was my first hint to test MSVC. – Josep Valls Feb 23 '12 at 14:57

How can I seekg() files over 4GB on Windows?

3 Answers3

Linked