1

My Java app reads in thousands of text files, goes through the lines and parse the text into floats and does some calculation with them then saves the results into some files, now I'm using multiple thread to execute them simultaneously, still need about an hour to run the whole app.

Here is a simplified pseudo version of what my app does :

String Lines[]=ReadFile("abc.txt"),Items[];
float its[];

for (int i=0;i<Lines.length;i++)
{
  Items=Lines[i].split(",");
  its=new float[Items.length];
  for (int j=0;j<its.length;j++) its[j]=Float.parseFloat(Items[j]);
  do some calculation with : its[]
}
Do more calculation, save results to a StringBuffer.
Save StringBuffer to ("abc_result.txt");

I've read that Rootbeer is able to shift some workload to GPU and speed up the process, so my questions are :

[1] Can Rootbeer handle text parsing and file I/O on the GPU ? Or do I need this portion done on CPU and only do the calculation on GPU ?

[2] Would my app benefit from Rootbeer ?

Frank
  • 28,342
  • 54
  • 158
  • 227

1 Answers1

1

The answer, as so often is: it depends.

Rootbeer will be able to shift some work to the GPU. But there are lots of caveats.

  1. You have to learn a whole new way of doing things.
  2. It might not speed it up. It rather depends on where the bottleneck in your application is. For instance, if your problem is I/O-bound, then GPU processing won't help. You can check this with your code as it stands, by looking at CPU usage while your app is running. If it's taking an hour, then unless your files are catastrophically huge, it does sound as though it's CPU-bound.
  3. (This is the most important caveat.) You should start by optimising the code that you have, and checking how much room there is for improvement. GPU processing is essentially just throwing more horsepower at the problem; which is a good idea if and only if your code is already running optimally.

Now, lots of the processing in your application, by the look of things, is to do with processing the strings. You're invoking .split() on a String, which breaks it up into a String[]; then you go through the bits and read each one to convert to a float; then you do your calculations. This is quite heavy work, especially on the garbage collector, which has to get rid of all these Strings.

I did some detailed analysis of something very similar, using integers instead of floating point, in this question, and it led to some detailed discussion. The conclusion there was that it made a huge difference to abandon the String-based approach and read the files in character by character, turning them manually into integers. Almost certainly something very similar will help you with producing floating point numbers.

So, in short, Rootbeer might help, but you have a lot you can do before you try something that drastic.

Community
  • 1
  • 1
chiastic-security
  • 19,689
  • 3
  • 33
  • 62