I am designing an application which takes files and split/merge them as per content and pushes it to some other system. Once processed no need of those files at all. So, I am planning to store files on HDD where application deployed instead of any distributed/network file system.
Why I haven't chosen network file system as in my scenarios I need to process huge files like 1GB and I am using JSON streams for processing those files. Also some times I need to use RandomAccessFile mechanism for splitting my content. If it happens over network file system, processing time could be high.
I thought about scaling the application as well with local file system, it can be scaled without any worry as target system expects processed data from the same system to which they pushed files.
Please provide your thoughts on this? I want to check I'm on right path