2

I am trying to hit some external API to fetch some data. When the data size is small, everything works fine but when the size of data returned by the API is big I get CONNECTION RESET exception. The below code is from java class InterfaceHelper and I have also marked the comment at the line no where I am getting exception [Its while trying to read data from InputStream].

I have tried to search so many question on STACKOVERFLOW itself but didn't find an appropriate answer. So please don't mark it as duplicate question. I want to know what is the reason behind this problem, and what is the proper solution for this problem. If you find this question as duplicate please answer it before marking it as duplicate. I dare you.

Find below my code which I have used. URL is some dummy url as for security reason I cannot mention the actual URL I have used.

 try{
        URL url = new URL("http://example.com/someParams/SOME-ACCESS-TOKEN");
        HttpURLConnection connection = (HttpURLConnection)url.openConnection();
        connection.setRequestMethod("GET");
        connection.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
        connection.setRequestProperty("Content-Language", "en-US");
        connection.setRequestProperty("X-EXAMPLE-LOGIN", "XXXXXXXX");
        connection.setRequestProperty("X-EXAMPLE-PASSWORD", "XXXXXX");
        connection.setUseCaches(false);
        connection.setDoInput(true);
        connection.setDoOutput(true);
        DataInputStream input = new DataInputStream(connection.getInputStream());
        String  ret = "";
        if(input!=null){
            for( int c = input.read(); c != -1; c = input.read() ) { //InterfaceHelper.java:695
                ret = ret + String.valueOf((char)c);
            }
        }

         if(input!=null)
             input.close();
         if(connection!=null)
             connection.disconnect();

                if(ret!=null && ret.length()>0){
                    return ret;
                }

    }catch(Exception e) {
        e.printStackTrace();
    }

This the exception I get

        java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(SocketInputStream.java:196)
        at java.net.SocketInputStream.read(SocketInputStream.java:122)
        at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
        at sun.security.ssl.InputRecord.readV3Record(InputRecord.java:554)
        at sun.security.ssl.InputRecord.read(InputRecord.java:509)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
        at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884)
        at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
        at sun.net.www.http.ChunkedInputStream.fastRead(ChunkedInputStream.java:244)
        at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:689)
        at java.io.FilterInputStream.read(FilterInputStream.java:133)
        at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3053)
        at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3047)
        at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3035)
        at java.io.FilterInputStream.read(FilterInputStream.java:83)
        at com.lsa.akosha.util.InterfaceHelper.hitApiBrandWatch(InterfaceHelper.java:695)
        at com.lsa.akosha.service.brand.BrandCronService.brandSentiments(BrandCronService.java:288)
        at com.lsa.akosha.util.thread.BrandWatchCronConverse.executeInternal(BrandWatchCronConverse.java:60)
        at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:113)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
Hitesh Kumar
  • 623
  • 1
  • 6
  • 21
  • try debugging your code and maybe you will find your problem – Sarthak Mittal Dec 02 '14 at 05:25
  • 1
    I have tried it but it always throws exception at "DataInputStream input = new DataInputStream(connection.getInputStream());" but in print stack trace it always show exception occur while it trying to read i.e. the for loop – Hitesh Kumar Dec 02 '14 at 05:31
  • Try this code on some other machine (or may be by turning of anti virus). – jsjunkie Dec 02 '14 at 05:38
  • The code you have posted cannot possibly produce this stacktrace. There is SSL in the stack trace and a plaintext HTTP URL. And `connection.setRequestProperty("Content-Type","application/x-www-form-urlencoded"` is also futile here as you aren't submitting any content. This cannot possibly be the real code. NB `connection` and `input` cannot possibly be null at the points you're testing them. Don't write futile tests. – user207421 Dec 02 '14 at 06:02

4 Answers4

1

maybe its just quirks, but I've found some "weird" things in your post.

  1. You use SSL connection (as it appears in the stacktrace) but in the provided example this is no HTTPS at all.
  2. application/x-www-form-urlencoded header is usually used with POST requests. Full explanation on this read here

I'm just saying that this can be misleading to some extent.

Now regarding the question itself.

All-in-all - connection reset means that for some reason the connection between the client and the server is broken. For example, the server decides to close the connection. The debugging process can be really tricky here, so if I were you, I would have tried a couple of techniques, depending on environment you're working on:

  1. Remove SSL and work with plain HTTP. This way you can put a proxy in-between and analyze the network traffic better. Stuff like Burp can help here.

  2. Try to eliminate the possibility that some firewall/proxy/whatever just dumps the connection for its own reasons. Maybe it worth to run this code in the same machine with the server (of course if its a viable option) just for the sake of testing how this works with "localhost", you know.

  3. I'm not really familiar with this fairly low level API and know all its quirks. But maybe you can use HttpClient instead, this way you'll probably eliminate the need to know all the "low-level" flags, maybe its something wrong there.

Hope this helps

Community
  • 1
  • 1
Mark Bramnik
  • 31,144
  • 4
  • 41
  • 69
0

connection.setRequestMethod("GET");

Instead get method try using

connection.setRequestMethod("POST");

because get method has limit to send the data.

Dhruv Raj Singh
  • 284
  • 1
  • 11
  • But he isn't sending any data; and calling `connection.setDoOutput(true);` sets the request method to `POST` anyway. – user207421 Dec 02 '14 at 06:11
  • I am reading data from server, I am not posting it so setting POST is not the real solution – Hitesh Kumar Dec 02 '14 at 06:41
  • @user2531799 So why exactly are you calling `setRequestMethod()` and setting the `"Content-Type"` if you're not sending any content? – user207421 Dec 02 '14 at 07:22
  • Because its a rest call, I have to tell the server that I am calling a rest GET service. And I have to tell the content type because rest service consumes and produces some content type. So this content type application/x-www-form-urlencoded is what this rest call consumes. Please refer this url for detailed info https://docs.oracle.com/cd/E19798-01/821-1841/gipzh/index.html – Hitesh Kumar Dec 02 '14 at 08:24
  • @user2531799 Make up your mind. *Either* 'I am not posting', which is what your first comment above says, *or* you *are* producing the data which is consumed, which is what you have just said in your second comment, which explains why the `POST` and the `Content-type`. You are presently contradicting yourself and you need to resolve it. Nobody else can do that for you. – user207421 Dec 02 '14 at 09:11
  • @EJP : right, so in my first comment I said I am not posting data I am reading means I am calling rest service to read data, I am not producing any data, Rest consumes and produces data. I am just reading what rest service produces data. I am providing some information to server i.e. I am just passing arguments for filter purpose and I am setting content type that what kind of data in filters I am sending. So rest will know what kind of filter I am sending to get the required data while content type say what how rest reads those parameters – Hitesh Kumar Dec 05 '14 at 04:38
0

This is clearly not the real code, as noted in my comment above, but

connection.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
connection.setDoOutput(true);

You aren't sending any output so you shouldn't be calling these methods at all. And setDoOutput(true) contradicts setRequestMethod("GET");.

Alternatively, send some output.

user207421
  • 289,834
  • 37
  • 266
  • 440
  • I am commenting on the code you posted. If you want comments on your real code, post it. And removing the security credentials code doesn't explain why you're calling the two methods I listed above. I would say your server has rejected your credentials, but we can only guess if you leave out relevant parts of your code. You don't need to post the actual credential values, just `xxx` and `yyy` will do. But you must post the real code. You haven't even posted an HTTPS URL. You're expecting a lot here. – user207421 Dec 02 '14 at 07:20
0

I tried to debug it again and again, this time while debugging I found that reading data from DataInputStream is the culprit here.

The line dataInputStream.read() reads one char at a time, and I have to convert it into String. Below is the code 

    String  ret = "";
    if(input!=null){
        for( int c = input.read(); c != -1; c = input.read() ) { //InterfaceHelper.java:695
            ret = ret + String.valueOf((char)c);
        }
    }

The main problem is that I am reading data into int and then convert it into char and then into String i.e. ret = ret + String.valueOf((char)c);.

Now I have removed the DataInputStream and used BufferInputStream and used bufferInputStream.readLine() which directly read's line in string from InputStream, so no type casting which saves lots of time in reading data from stream and hence till the time connection and stream expires it reads the data and then I close both.

This way my problem is resolved. I am posting my new code which helps me in solving my problem.

   try{
        URL url = new URL("http://example.com/someParams/SOME-ACCESS-TOKEN");
        HttpURLConnection connection = (HttpURLConnection)url.openConnection();
        connection.setRequestMethod("GET");
        connection.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
        connection.setRequestProperty("Content-Language", "en-US");
        connection.setRequestProperty("X-EXAMPLE-LOGIN", "XXXXXXXX");
        connection.setRequestProperty("X-EXAMPLE-PASSWORD", "XXXXXX");
        connection.setUseCaches(false);
        connection.setDoInput(true);
        connection.setDoOutput(true);
        connection.setConnectTimeout(99999999);
        connection.setReadTimeout(99999999);
        BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        //DataInputStream input = new DataInputStream(connection.getInputStream());
        String  ret = "";
       /*   if(in!=null){
            for( int c = input.read(); c != -1; c = input.read() ) {
                ret = ret + String.valueOf((char)c);
                if(input==null || connection==null)
                    break;
            }
       }*/
       String inputLine;
       while ((inputLine = in.readLine()) != null) 
       {
            ret = ret + inputLine;
       }
       if(in!=null)
         in.close();
       if(connection!=null)
         connection.disconnect();
       if(ret!=null && ret.length()>0){
           return ret;
       }

   }catch(Exception e) {
      e.printStackTrace();
   }
Hitesh Kumar
  • 623
  • 1
  • 6
  • 21
  • It isn't. None of that would cause a connection reset. – user207421 Dec 04 '14 at 21:07
  • @EJP: I am reading data from inputstream right. When I was reading it from DataInputStream I am reading every character and converting into string which took time in typecasting but when I used bufferReader I am reading whole line, which directly reads data in string, no typecasting or typecasting is done internally, i.e. no manual typecasting. So it resolves my problem. The main reason behind Connection reset is I am reading data from Stream which is already closed and the reason behind close is its timeout while reading data from stream – Hitesh Kumar Dec 05 '14 at 04:47
  • on your case the part which caues the problem was the connection.getInputStream(), not the readers wrapped. Your problem was resolved becasue of another thing but the solution is not changing from DataInputStream to BufferedReader because the socket exceptions occurs earlier when opening the stream. – benchpresser Sep 11 '20 at 23:04