3

First, this question looks like Very large SOAP response - Android- out of memory error subject. Because of my English weakness and the similarity of this problem, with a view to facilitate understanding some statement passages were copied.

I have an application where i need to download a large amount of data via a SOAP call to a webservice. The response is then sent to a function which display the XML file.

The data is more than 11MB in size and i have a java.lang.OutOfMemoryError everytime.

Modifying the webservice to give out smaller amounts of data is not an option.

I use "Http request" to get datas. I know that : my resquest is fine, soapUi and wireshark return expected responses.

But my AVD isn't able to pass this line

HttpResponse httpResponse = httpClient.execute(httpPost);

After some minutes of work (during which wireshark recovers some expected queries) this error is made

 Out of memory on a 16705124-byte allocation.

I tried to upgrade the SD card size to 20GB, yet, error still.

Parsing httpResponse is probably the next step, is it possible to parse HttpResponse while it receives data by fragmenting it into several parts by exmple?

Do you have an idea ?

Thanks, Dsandre

Community
  • 1
  • 1
Dsandre
  • 312
  • 1
  • 5
  • 14

3 Answers3

4

Thank's Graeme, opening connection as a byte stream works. If it could be helpfull to somebody, this is my source code

        //Make a temporary file to save data
        File responseFile = File.createTempFile("SOAP", "xml", context.getFilesDir());      
        int nbCharRead = 0; int i=0; int totalRead = 0;

        //Send query
        OutputStream outputS = url_Connection.getOutputStream();
        Writer w_out = new OutputStreamWriter(outputS);
        w_out.write(webServiceXml);
        w_out.flush();
        w_out.close();


        //Buffers
        BufferedReader bufReader = new BufferedReader(new InputStreamReader(url_Connection.getInputStream()));
        BufferedWriter bufWriter = new BufferedWriter(new FileWriter(responseFile));
        char[] buffer = new char[10000];



        while((nbCharRead = bufReader.read(buffer, 0, 10000)) != -1)
        {
            totalRead += nbCharRead;
            Log.d("Test InputStream", "i: " + i++ +" - " + nbCharRead + " -> " + totalRead);
            bufWriter.write(buffer, 0, nbCharRead );
        }       

        if(bufWriter != null)
        {
            bufWriter.flush();
            bufWriter.close();
        }


        Log.w(MsgLog, "--- Stream Got--- ; Total : " + totalRead);
Dsandre
  • 312
  • 1
  • 5
  • 14
  • Can u please tell me what is webServiceXml.How can i replace this line of code androidHttpTransport.call(SOAP_ACTION_async, envelope); to get this working.Since i get the error on this line when calling the webservice.Any help will be appreciable.Referred to other links as well but no proper code available. – Raj Jul 03 '13 at 06:50
3

You can open a connection as a byte stream:

URL url = new URL("www.google.com");
InputStream in = new BufferedInputStream(url.openStream());
byte[] buffer = new byte[1024];
int chunk = 0;
while ((chunk =in.read(buffer ))!=-1)
{
    proccessChunk(chunk);
}
in.close();

From this point you should be able to metre the response and process it bit-by-bit, saving each processed chunk to text files (or, if it is relational data to SQLite).

HTH

Graeme
  • 24,857
  • 23
  • 121
  • 182
  • Your solution works, I get data with `InputStream`. Thank you. But now a newest problem arose: **I can't get more than 28847 bytes**, why? I tried many options (with/without timeout,..). The stream is stored on the buffer (`byte[104876]` like [Very large SOAP response - Android- out of memory error](http://stackoverflow.com/questions/4941581/very-large-soap-response-android-out-of-memory-error/6925619#6925619) and the buffer is write on a tempory file. – Dsandre Aug 04 '11 at 16:36
1

You might be loading the entire response into memory(maybe as a string?). Android devices, especially older ones running on older hardware and older versions of the os, have a limited vm heap size. I can't remember the exact number but a quick google indicates it is as low as 16mb.

The best way to process responses that large in android devices is to read it as a stream. I think the latest versions of apache-commons/jakarta httpclient supports streaming.

Hyangelo
  • 4,644
  • 4
  • 22
  • 33
  • I tried to do this thank's to second option of this : [1.1.5. Ensuring release of low level resources](http://hc.apache.org/httpcomponents-client-ga/tutorial/html/fundamentals.html#d4e122). But because of the HttpResponse failure I can't create the HttpEntity to stream – Dsandre Aug 03 '11 at 13:39
  • Can you try using executeMethod instead of execute. See: http://hc.apache.org/httpclient-3.x/performance.html – Hyangelo Aug 03 '11 at 21:02
  • The problem with the execute() method is that if you trace HttpClient.execute is that it already starts reading and processing the reponse so your allocation error might be caused by that. I am not sure though, try the executeMethod and see if you are able to get past executeMethod and proceed to reading the response stream. – Hyangelo Aug 03 '11 at 21:10
  • Thank's. _executeMethod_ isn't found, the httpClient type is 'DefaultHttpClient'. I chose it because org.apach.http.client.HttpClient is an interface, and I don't know how implement my own httpClient class – Dsandre Aug 04 '11 at 09:28