Your bottleneck is in the I/O. You want the program to read in as much data into memory in fewest I/O calls. For example reading 256 numbers with one fread
is faster than 256 fread
of one number.
If you can, format the data file to match the target platform's internal floating point representation, or at least your program's representation. This reduces the overhead of translating textual representation to internal representation.
Bypass the OS and use the DMA controller to read in the file data, if possible. The DMA chip takes the burden of reading data into memory off the shoulders of the processor.
Compact you data file. The data file wants to be in one contiguous set of sectors on the disk. This will reduce the amount of time spent seeking to different areas on the physical platters.
Have you program demand exclusive control over the disk resource and the processors. Block all other unimportant tasks; raise the priority of your program's execution.
Use multiple buffers to keep the disk drive spinning. A large portion of time is spent waiting for the hard drive to accelerate and decelerate. Your program can be processing the data while something else is storing the data into a buffer, which leads to ...
Multi-thread. Create one thread to read in the data and alert the processing task when the buffer is not empty.
These should keep you busy for a while. All other optimizations will result in negligible performance gains. (Such as accessing the hard drive controller directly to transfer into one of your buffers.)