Volley - parseNetworkResponse OutOfMemoryError

Question

I am using Volley to manage my app. Sometimes when a response is to large, I will get an OOM on devices with no much RAM. I am unsure how to resolve this. I understand Volley stores its responses in memory but I have my app wrapped around Volley so much that it would be a pain to switch. I had issues using Retrofit too. I have tried using a JsonReader but it still seems to happen. I have a custom request used with Volley. It returns a Gson JsonObject. Here is my code currently, the app is receiving an OOM when the response data length is returning 5511532 (5mb).

@Override
protected Response<T> parseNetworkResponse(NetworkResponse response) {
    try {
        String json = new String(response.data, HttpHeaderParser.parseCharset(response.headers));

        if (LOG_HEADERS) {
            Log.d("GsonResponseHeaders", response.headers.toString());
        }
        if (LOG_JSON) {
            Log.d("GsonRequestResponse", json);
        }

        return Response.success(gson.fromJson(json, clazz), HttpHeaderParser.parseCacheHeaders(response));
    } catch (UnsupportedEncodingException e) {
        return Response.error(new ParseError(e));
    } catch (JsonSyntaxException e) {
        return Response.error(new ParseError(e));
    } catch (OutOfMemoryError e) {
        Log.d("ResponseLength", "-- " + response.data.length);
        return Response.error(new ParseError(e));
    }
}

EDIT:

Here is a sample response that throws OOM.

GsonRequestResponse: {"user_company_info":[{"id":"1","company_id":"5","circle_id":"34","status":"A","cat_id":"1","company_map":"0","timestamp":"2015-07-09 15:28:40","added_by":"1","add_date":"0000-00-00","name":"FSM","added_datetime":"2014-02-12 16:16:51","clause":"fsm_company","company_name":"Assero Services LLC - FSM","circles":[{"id":"1","company_id":"5","circle_id":"34","status":"A","cat_id":"1","company_map":"0","timestamp":"2015-07-09 15:28:40","added_by":"1","add_date":"0000-00-00","circle_name":"Cityside - HUD Contract Area - 5D","create_by":"2536","create_date":"0000-00-00","circle_owner":"0","circle_owner_company":"26792","circle_company_identifier":"0","level":"OUTER","assign_fsm":"26792","assign_client":"25491","assign_am":"0","routine_1":"10065","routine_2":"10209","routine_3":"10062","routine_4":"0","image_duplicate_api":"Y","routine_1_type":null,"routine_1_method":null,"routine_1_cancel_buffer":null,"routine_2_type":null,"routine_2_method":null,"routine_2_cancel_buffer":null,"routine_3_type":null,"routine_3_method":null,"routine_3_cancel_buffer":null,"routine_4_type":null,"routine_4_method":null,"routine_4_cancel_buffer":null,"name":"FSM","added_datetime":"2014-02-12 16:16:51","clause":"fsm_company","can_issue":"1"},{"id":"1","company_id":"5","circle_id":"33","status":"A","cat_id":"1","company_map":"0","timestamp":"2015-07-09 15:28:40","added_by":"1","add_date":"0000-00-00","circle_name":"Cityside - HUD Contract Area - 4D","create_by":"2536","create_date":"0000-00-00","circle_owner":"0","circle_owner_company":"26792","circle_company_identifier":"0","level":"OUTER","assign_fsm":"26792","assign_client":"25491","assign_am":"0","routine_1":"10065","routine_2":"10209","routine_3":"10062","routine_4":"0","image_duplicate_api":"Y","routine_1_type":null,"routine_1_method":null,"routine_1_cancel_buffer":null,"routine_2_type":null,"routine_2_method":null,"routine_2_cancel_buffer":null,"routine_3_type":null,"routine_3_method":null,"routine_3_cancel_buffer":null,"routine_4_type":null,"routine_4_method":null,"routine_4_cancel_buffer":null,"name":"FSM","added_datetime":"2014-02-12 16:16:51","clause":"fsm_company","can_issue":"2"},{"id":"1","company_id":"5","circle_id":"32","status":"A","cat_id":"1","company_map":"0","timestamp":"2015-07-09 15:28:40","added_by":"1","add_date":"0000-00-00","circle_name":"Cityside - HUD Contract Area - 2D","create_by":"2536","create_date":"0000-00-00","circle_owner":"0","circle_owner_company":"26792","circle_company_identifier":"0","level":"OUTER","assign_fsm":"26792","assign_client":"25491","assign_am":"0","routine_1":"10065","routine_2":"10209","routine_3":"0","routine_4":"0","image_duplicate_api":"Y","routine_1_type":null,"routine_1_method":null,"routine_1_cancel_buffer":null,"routine_2_type":null,"routine_2_method":null,"routine_2_cancel_buffer":null,"routine_3_type":null,"routine_3_method":null,"routine_3_cancel_buffer":null,"routine_4_type":null,"routine_4_method":null,"routine_4_cancel_buffer":null,"name":"FSM","added_datetime":"2014-02-12 16:16:51","clause":"fsm_company","can_issue":"1"}

EDIT:

Hey guys, so I tried splitting up my responses. It seemed to help but after 3-4 requests it still gives me an OOM. I believe it has something to do with Volley. Here is my GC for each response, it grows each time. Notice that the heap grows each time. I put some spacing between those so you can see the heap log.

10-12 06:52:21.236 23515-23515/com.droid.visneta D/RequestQueue: batch_open_orders_request started... 10-12 06:52:21.236 23515-23855/com.droid.visneta D/GsonCustomHeaders: {UserSession=source=Android-ffffffff-83d1-3e36-ffff-ffff99d603a9,id=6,token=9eb7c26405defb48fe884cfef8000696} 10-12 06:52:21.236 23515-23855/com.droid.visneta I/qtaguid: Failed write_ctrl(t -1 4883184842671390720 -1) res=-1 errno=9 10-12 06:52:21.236 23515-23855/com.droid.visneta I/qtaguid: Tagging socket -1 with tag 43c48ebe00000000(1136955070) for uid -1 failed errno=-9 10-12 06:52:21.236 23515-23855/com.droid.visneta I/NetworkManagementSocketTagger: tagSocketFd(-1, 1136955070, -1) failed with errno-9 10-12 06:52:21.936 23515-23855/com.droid.visneta I/System.out: Removed SSLv3 from enabled protocols 10-12 06:52:21.976 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 1990K, 15% free 33771K/39495K, paused 11ms+1ms, total 38ms 10-12 06:52:29.940 23515-23855/com.droid.visneta I/qtaguid: Failed write_ctrl(u -1) res=-1 errno=9 10-12 06:52:29.940 23515-23855/com.droid.visneta I/qtaguid: Untagging socket -1 failed errno=-9 10-12 06:52:29.940 23515-23855/com.droid.visneta W/NetworkManagementSocketTagger: untagSocket(-1) failed with errno -9 10-12 06:52:31.808 23515-23855/com.droid.visneta D/dalvikvm: GC_FOR_ALLOC freed 747K, 14% free 34267K/39495K, paused 18ms, total 18ms 10-12 06:52:32.616 23515-23855/com.droid.visneta D/dalvikvm: GC_FOR_ALLOC freed 596K, 12% free 34778K/39495K, paused 21ms, total 21ms 10-12 06:52:32.616 23515-23855/com.droid.visneta I/dalvikvm-heap: Grow heap (frag case) to 36.423MB for 2093068-byte allocation 10-12 06:52:32.648 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 10K, 12% free 36811K/41543K, paused 10ms+1ms, total 29ms 10-12 06:52:32.648 23515-23855/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 9ms 10-12 06:52:33.272 23515-23855/com.droid.visneta D/dalvikvm: GC_FOR_ALLOC freed 1122K, 14% free 35790K/41543K, paused 20ms, total 20ms 10-12 06:52:33.272 23515-23855/com.droid.visneta D/Volley: [212] BasicNetwork.logSlowRequests: HTTP response for request=<[ ] https://dev.visneta.com/amf/gateway?contentType=application/json 0x43c48ebe NORMAL 2> [lifetime=12039], [size=1652746], [rc=200], [retryCount=0] 10-12 06:52:33.296 23515-23855/com.droid.visneta D/dalvikvm: GC_FOR_ALLOC freed 2048K, 15% free 35360K/41543K, paused 22ms, total 22ms

10-12 06:52:33.296 23515-23855/com.droid.visneta I/dalvikvm-heap: Grow heap (frag case) to 38.149MB for 3305504-byte allocation

10-12 06:52:33.340 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 1K, 14% free 38586K/44807K, paused 11ms+11ms, total 43ms 10-12 06:52:33.392 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 555K, 11% free 40078K/44807K, paused 11ms+0ms, total 33ms 10-12 06:52:33.392 23515-23855/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 9ms 10-12 06:52:33.448 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 558K, 8% free 41568K/44807K, paused 11ms+1ms, total 36ms 10-12 06:52:33.448 23515-23855/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 15ms 10-12 06:52:33.504 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 564K, 4% free 43050K/44807K, paused 12ms+1ms, total 38ms 10-12 06:52:33.504 23515-23855/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 15ms 10-12 06:52:33.560 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 561K, 3% free 44537K/45575K, paused 10ms+1ms, total 38ms 10-12 06:52:33.560 23515-23855/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 8ms 10-12 06:52:33.616 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 577K, 3% free 46056K/47111K, paused 11ms+0ms, total 38ms 10-12 06:52:33.624 23515-23855/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 9ms 10-12 06:52:33.676 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 550K, 3% free 47555K/48583K, paused 11ms+1ms, total 38ms 10-12 06:52:33.676 23515-23855/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 16ms 10-12 06:52:33.764 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 4602K, 11% free 45006K/50119K, paused 11ms+12ms, total 58ms 10-12 06:52:33.840 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 2055K, 11% free 44998K/50119K, paused 10ms+0ms, total 45ms 10-12 06:52:33.840 23515-24293/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 26ms 10-12 06:52:33.916 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 2057K, 11% free 44988K/50119K, paused 11ms+1ms, total 43ms 10-12 06:52:33.916 23515-24293/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 21ms 10-12 06:52:33.988 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 2044K, 11% free 44991K/50119K, paused 11ms+1ms, total 41ms 10-12 06:52:33.988 23515-24293/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 21ms 10-12 06:52:34.056 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 2037K, 11% free 45002K/50119K, paused 11ms+1ms, total 41ms 10-12 06:52:34.060 23515-24293/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 8ms 10-12 06:52:34.132 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 2050K, 11% free 44995K/50119K, paused 11ms+0ms, total 42ms 10-12 06:52:34.132 23515-24293/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 8ms 10-12 06:52:34.216 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 2017K, 11% free 45026K/50119K, paused 11ms+11ms, total 53ms 10-12 06:52:34.216 23515-24293/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 2ms 10-12 06:52:34.284 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 2067K, 11% free 45004K/50119K, paused 11ms+0ms, total 40ms 10-12 06:52:34.288 23515-24293/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 23ms 10-12 06:52:34.372 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 2028K, 11% free 45000K/50119K, paused 11ms+12ms, total 54ms 10-12 06:52:34.452 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 2036K, 11% free 45012K/50119K, paused 12ms+1ms, total 48ms 10-12 06:52:34.452 23515-24293/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 4ms 10-12 06:52:34.524 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 2056K, 11% free 45003K/50119K, paused 12ms+0ms, total 41ms 10-12 06:52:34.524 23515-24293/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 22ms 10-12 06:52:34.596 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 2042K, 11% free 45004K/50119K, paused 11ms+1ms, total 43ms 10-12 06:52:34.600 23515-24293/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 18ms 10-12 06:52:34.668 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 2041K, 11% free 45009K/50119K, paused 11ms+1ms, total 42ms 10-12 06:52:34.668 23515-24293/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 21ms 10-12 06:52:34.744 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 2049K, 11% free 45007K/50119K, paused 11ms+1ms, total 41ms 10-12 06:52:34.744 23515-24293/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 19ms 10-12 06:52:34.776 23515-24293/com.droid.visneta D/BatchOpenOrders: 2000 open wos batched. 10-12 06:52:34.776 23515-23515/com.droid.visneta D/Routine Services Count: 1411 10-12 06:52:34.776 23515-23515/com.droid.visneta D/Open Order Count: 2000 10-12 06:52:37.288 23515-23515/com.droid.visneta D/RequestQueue: batch_open_orders_request started... 10-12 06:52:37.288 23515-23856/com.droid.visneta D/GsonCustomHeaders: {UserSession=source=Android-ffffffff-83d1-3e36-ffff-ffff99d603a9,id=6,token=9eb7c26405defb48fe884cfef8000696} 10-12 06:52:37.288 23515-23856/com.droid.visneta I/qtaguid: Failed write_ctrl(t -1 4883184842671390720 -1) res=-1 errno=9 10-12 06:52:37.288 23515-23856/com.droid.visneta I/qtaguid: Tagging socket -1 with tag 43c48ebe00000000(1136955070) for uid -1 failed errno=-9 10-12 06:52:37.288 23515-23856/com.droid.visneta I/NetworkManagementSocketTagger: tagSocketFd(-1, 1136955070, -1) failed with errno-9 10-12 06:52:37.592 23515-23856/com.droid.visneta I/System.out: Removed SSLv3 from enabled protocols 10-12 06:52:42.612 23515-23856/com.droid.visneta I/qtaguid: Failed write_ctrl(u -1) res=-1 errno=9 10-12 06:52:42.616 23515-23856/com.droid.visneta I/qtaguid: Untagging socket -1 failed errno=-9 10-12 06:52:42.616 23515-23856/com.droid.visneta W/NetworkManagementSocketTagger: untagSocket(-1) failed with errno -9 10-12 06:52:42.924 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 1779K, 10% free 45206K/50119K, paused 11ms+1ms, total 42ms 10-12 06:52:43.724 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 1104K, 9% free 46034K/50119K, paused 10ms+0ms, total 39ms 10-12 06:52:43.816 23515-23856/com.droid.visneta D/dalvikvm: GC_FOR_ALLOC freed 15K, 9% free 46033K/50119K, paused 28ms, total 28ms 10-12 06:52:43.816 23515-23856/com.droid.visneta I/dalvikvm-heap: Grow heap (frag case) to 47.417MB for 2095116-byte allocation 10-12 06:52:43.852 23515-23518/com.droid.visneta D/dalvikvm: GC_CONCURRENT freed 8K, 8% free 48071K/52167K, paused 12ms+1ms, total 38ms 10-12 06:52:43.852 23515-23856/com.droid.visneta D/dalvikvm: WAIT_FOR_CONCURRENT_GC blocked 16ms 10-12 06:52:44.312 23515-23856/com.droid.visneta D/dalvikvm: GC_FOR_ALLOC freed 1108K, 10% free 47049K/52167K, paused 26ms, total 26ms 10-12 06:52:44.312 23515-23856/com.droid.visneta I/dalvikvm-heap: Grow heap (frag case) to 47.922MB for 1585235-byte allocation 10-12 06:52:44.340 23515-23856/com.droid.visneta D/dalvikvm: GC_FOR_ALLOC freed 0K, 10% free 48597K/53767K, paused 25ms, total 25ms 10-12 06:52:44.340 23515-23856/com.droid.visneta D/Volley: [213] BasicNetwork.logSlowRequests: HTTP response for request=<[ ] https://dev.visneta.com/amf/gateway?contentType=application/json 0x43c48ebe NORMAL 3> [lifetime=7052], [size=1585223], [rc=200], [retryCount=0] 10-12 06:52:44.364 23515-23856/com.droid.visneta D/dalvikvm: GC_FOR_ALLOC freed 2050K, 14% free 46551K/53767K, paused 26ms, total 26ms

10-12 06:52:44.368 23515-23856/com.droid.visneta I/dalvikvm-heap: Grow heap (frag case) to 48.949MB for 3170458-byte allocation

`...when the response data length is returning 5511532...` what? bits? bytes? kilobytes? megabytes? Writte the measured unit. I suggest you use each file with up to 250 kilobytes each, since Over the Air networks tend to loose connection, and have worse transmittion than cables.... Finally, are all of those info necessary at the same time and in all requests? If you particionate the info and segregate by "meaning", then users can see the info, while not downloading irrelevant info. — Bonatti, Oct 08 '15 at 11:27

score 0 · Answer 1 · answered Oct 07 '15 at 20:11

0

The problem is not with Volley or Retrofit it is most likely to do with the size or the data response. If you have control over the web service you should look at breaking down the information in to smaller chunks by using paging.

answered Oct 07 '15 at 20:11

Ivan Wooll

3,290
2
19
26

Is it possible to use JsonReader to fix it or what if I split up the response into chunks rather than changing the server side at all? – Joe Ginley Oct 07 '15 at 20:13
If you use JsonReader or any other client side approach you're still going to have to load the data in to memory which is what is causing your problem. Do you have sample of the response you could share? – Ivan Wooll Oct 07 '15 at 20:15
Is that the whole response? Send it to me in an email and I'll be able to give you a better answer. Email's in my profile. – Ivan Wooll Oct 07 '15 at 20:23
No that is what is logged into the logcat. Alright, I will thanks. – Joe Ginley Oct 07 '15 at 20:24
Adding largeHeap to my manifest fixes the issue for me but I would prefer not to take that route. – Joe Ginley Oct 07 '15 at 20:24

score 0 · Accepted Answer · edited May 23 '17 at 11:59

0

The response is pretty big (over 5 Mb) and it makes sense that weak devices do not have enough RAM for it. It's a tough work, but if the server-side is managed by you, you should split this response into few responses. Sending it in chunks wouldn't work since you still have to piece them together before handling the response, and anyway I don't think that you should parse so much data in one request.

I also suggest you to check out this answer, it will help you find the limits of the device you check: https://stackoverflow.com/a/9428660/5280641.

edited May 23 '17 at 11:59

Community

1
1

answered Oct 07 '15 at 20:26

Neria Nachum

1,469
1
17
31

thanks, I will check out that link. I am managing the server side but the iOS app does not have this issue. I guess I could create new functions just for the Android app and then convert iOS to use the new functions later? – Joe Ginley Oct 07 '15 at 20:39
As far as I read, iPhones, including old version, have a pretty large heap. Anyway, that could be a good temporary solution. Good luck! – Neria Nachum Oct 07 '15 at 20:54
Should I try to avoid using largeHeap? – Joe Ginley Oct 08 '15 at 04:19

BNK · Answer 3 · 2015-10-08T00:06:26.203

If you have control over your server side, IMO, response data compression is another working way for such issues. Of course, in client side, you will have to decompress the network response first, then parse as normal.

If your server app is ASP.NET Web API, for example, you can take a look at this ASP.NET Web API Compression documentation for reference. I think other web services or some Web servers also support compression.

Hope this helps!

score 0 · Answer 4 · edited May 23 '17 at 10:28

0

Well i don't know much about volley but i've handled this kind of json response with Jackson lib.

Check this answers Android: Parsing large JSON file

I think Jackson stream API is the way to go.

edited May 23 '17 at 10:28

Community

1
1

answered Oct 12 '15 at 07:07

Kalem

1,122
12
31

Volley - parseNetworkResponse OutOfMemoryError

4 Answers4