efficient input stream to string method in java

Question

So I was running a profiler on my (admittedly pretty simple) application in Java, and was surprised that second only to methods that required making HTTP requests in terms of time was my inputStreamToString method. It's currently defined like this:

public static String inputStreamToString(InputStream in) throws IOException {
    BufferedReader reader = new BufferedReader(new InputStreamReader(in));
    StringBuilder sb = new StringBuilder();
    String line = null;
    while ((line = reader.readLine()) != null) {
        sb.append(line).append("\n");
    }
    in.close();
    return sb.toString();
}

How can I make this faster? (And yes, I really do need strings, and no, the InputStrings aren't that large, and no, this method is being called less frequently than most methods in the program, and no, I there is no way to avoid the need of conversion.)

Well, that's where all the I/O happens (I assume that the profiler includes all the time readLine() takes to wait for the data to come in). The only obvious thing you could do is to preinitialize the StringBuilder with a sufficiently large buffer so it doesn't have to reallocate memory, but I suppose that everything is dwarved by the time it takes to read data. — EboMike, Nov 26 '10 at 21:15
@EboMike has pretty much nailed it there. Apart from that, the only other thing I can think of is to skip the `BufferedReader` and supply your own buffer to read in to -- that would also save the `readLine` calls since you really don't care about reading one line at a time. — casablanca, Nov 26 '10 at 21:19
Unless you want to convert line breaks or parse lines it does not make a lot of sense to read line by line. I would rather read char by char via fixed size buffers: gist.github.com/fkirc/a231c817d582e114e791b77bb33e30e9 — Mike76, Jul 03 '19 at 11:20

score 5 · Accepted Answer · answered Nov 26 '10 at 21:21

Well, that's where all the I/O happens (I assume that the profiler includes all the time readLine() takes to wait for the data to come in). The only obvious thing you could do is to preinitialize the StringBuilder with a sufficiently large buffer so it doesn't have to reallocate memory, but I suppose that everything is dwarved by the time it takes to read data.

Other than that - you're I/O bound. It simply takes time to receive data through the network.

EDIT: May as well include casablanca's comment too: Instead of reading line-by-line and then adding a newline, you may as well use a simple reader with a reasonably big buffer that you provide and just block-read everything. There's no need to read line-by-line since you seem to just copy the entire input data anyway. The only reasoning for manually going line-by-line is if you wanted to normalize the newlines (like \r\n) to a standard \n.

score 0 · Answer 2 · answered Nov 26 '10 at 21:25

Try to use IOUtils.copy() from jakarta commons. Create ByteArrayOutputStream, copy the bytes from the HTTPRequest stream to this ByteArray, then create string using new String(bytes, "UTF-8").

I believe it can be faster...

But your code looks like it is written to illustrate good style and good coding at all. I really do not understand what may be so ineffective here. Probably it takes time because the rest of your logic is relatively simple and written well? I mean probably although this piece of code takes relatively much time this is not too critical?

efficient input stream to string method in java

2 Answers2

Linked