I have Google'd my butt off, and I can't find anything on this topic.
I am trying to create a download client using Java, and I have figured out how to download files with Java, but I want to accelerate the download speed. I know how this works (opening several connections to the download server), but how can I achieve this?
I am looking for either some detailed explanation of such an algorithm or some code examples.
This is only possible if the server side supports range requests. You can determine that by checking using a HEAD request if the HTTP response header contains Accept-Ranges: bytes. If that is the case, then you can just spawn several threads which downloads the file in parts using the Range header. The URLConnection and ExecutorService are helpful in this.
Keep in mind that you also take the limitation in amount of threads and network bandwidth of your own machine into account.
BalusC described the trick and here is a reference to some source-code you can review and start with:
JDownLoader[Java]: http://svn.jdownloader.org/projects/show/jd
Free Download Manager[CPP]: http://freedownload.svn.sourceforge.net/viewvc/freedownload/
@BalusC Nice Work
I'm a bit unclear, are you writing a Java client that will talk to a server (perhaps a Java servlet?), so you control both sides of the data transfer? If so, you can do nearly anything you want. Java has java.util.zip, which has functions to do the compression.
If you want to download four (or N) files at once, just start up N threads and pass the HTTP requests to the server in parallel. This may not actually improve things, depending on link speed, network congestion, etc.
Writing your own client and making it properly multi-thread safe is a whole lot of work, which is why people just use the Apache HTTP client code. Its rock solid.
