2

I am fetching a page with a lot of xml. When I say a lot, I mean a little in terms of computing (~600 KB). For a reason I am unaware of, running EntityUtils.toString(entity) to turn the xml document (httpGet -> URL -> XML) takes about a half hour which is ridiculous on many levels.

I think that there should be a better way to do this. Can anyone fill me in?

Goal:

Alternative (Faster) way to do 

HttpGet httpGet = new HttpGet(urlLocation);
HttpResponse response = httpClient.execute(httpGet);
HttpEntity entity = response.getEntity();

String result = EntityUtils.toString(entity);  // <--- This guy

httpGet.releaseConnection();
bneigher
  • 778
  • 4
  • 10
  • 23
  • If you make the request in a browser, how long does it take? What is the value of `entity.getContentLength()`? – Sotirios Delimanolis Feb 14 '14 at 00:42
  • @SotiriosDelimanolis the request in the browser takes about 20 seconds to begin streaming the contents (slow but its fine), and to completely load the content takes about 5 mins. In the code this process takes about 25 mins – bneigher Feb 14 '14 at 00:46
  • `EntityUtils.toString(HttpEntity)` simply reads from the underlying connection's `InputStream`, which I believe reads from a `SocketInputStream`. – Sotirios Delimanolis Feb 14 '14 at 00:54
  • @bneigher do you remember what was a problem? I have same problem now and I have no idea what can be the reason. Response size is 97MB some big query from ES response.getEntity() : [Content-Length: 97775263,Chunked: false]. It is just stacked on the EntityUtils.toString(response.getEntity()); line without exception without anything. It runs on batch process, and it was always on 5 min, but it always starts all over and can not pass throw this line. I set it on 8 min but even that is not enough. – mommcilo May 26 '20 at 11:21

0 Answers0