5

I've written some code for my Android device to login to a web site over https and parse some data out of the resulting pages. An HttpGet happens first to get some info needed for login, then an HttpPost to do the actual login process.

The code below works great in a Java project within Eclipse which has the following Jar files on the build path: httpcore-4.1-beta2.jar, httpclient-4.1-alpha2.jar, httpmime-4.1-alpha2.jar, commons-logging-1.1.1.jar.

public static MyBean gatherData(String username, String password) {
    MyBean myBean = new MyBean();
    try {
        HttpResponse response = doHttpGet(URL_PAGE_LOGIN, null, null);
        System.out.println("Got login page");
        String content = EntityUtils.toString(response.getEntity());
        String token = ContentParser.getToken(content);
        String cookie = getCookie(response);
        System.out.println("Performing login");
        System.out.println("token = "+token +" || cookie = "+cookie);
        response = doLoginPost(username,password,cookie, token);
        int respCode = response.getStatusLine().getStatusCode();
        if (respCode != 302) {
            System.out.println("ERROR: not a 302 redirect!: code is \""+ respCode+"\"");
            if (respCode == 200) {
                System.out.println(getHeaders(response));
                System.out.println(EntityUtils.toString(response.getEntity()).substring(0, 500));
            }
        } else {
            System.out.println("Logged in OK, loading account home");
            // redirect handler and rest of parse removed
        }
    }catch (Exception e) {
        System.out.println("ERROR in gatherdata: "+e.toString());
        e.printStackTrace();
    }
    return myBean;
}
private static HttpResponse doHttpGet(String url, String cookie, String referrer) {
    try {
        HttpClient client = new DefaultHttpClient();
        client.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
        client.getParams().setParameter(CoreProtocolPNames.HTTP_CONTENT_CHARSET, "UTF-8");
        HttpGet httpGet = new HttpGet(url);
        httpGet.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
        httpGet.setHeader(HEADER_USER_AGENT,HEADER_USER_AGENT_VALUE);
        if (referrer != null && !referrer.equals("")) httpGet.setHeader(HEADER_REFERER,referrer);
        if (cookie != null && !cookie.equals("")) httpGet.setHeader(HEADER_COOKIE,cookie);
        return client.execute(httpGet);
    } catch (Exception e) {
        e.printStackTrace();
        throw new ConnectException("Failed to read content from response");
    }
}
private static HttpResponse doLoginPost(String username, String password, String cookie, String token) throws ClientProtocolException, IOException {
    try {
        HttpClient client = new DefaultHttpClient();
        client.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
        client.getParams().setParameter(CoreProtocolPNames.HTTP_CONTENT_CHARSET, "UTF-8");
        HttpPost post = new HttpPost(URL_LOGIN_SUBMIT);
        post.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
        post.setHeader(HEADER_USER_AGENT,HEADER_USER_AGENT_VALUE);
        post.setHeader(HEADER_REFERER, URL_PAGE_LOGIN);
        post.setHeader(HEADER_COOKIE, cookie);
        post.setHeader("Content-Type","application/x-www-form-urlencoded");
        List<NameValuePair> formParams = new ArrayList<NameValuePair>();
        formParams.add(new BasicNameValuePair("org.apache.struts.taglib.html.TOKEN", token));
        formParams.add(new BasicNameValuePair("showLogin", "true"));
        formParams.add(new BasicNameValuePair("upgrade", ""));
        formParams.add(new BasicNameValuePair("username", username));
        formParams.add(new BasicNameValuePair("password", password));
        formParams.add(new BasicNameValuePair("submit", "Secure+Log+in"));
        UrlEncodedFormEntity entity = new UrlEncodedFormEntity(formParams,HTTP.UTF_8);
        post.setEntity(entity);
        return client.execute(post);
    } catch (Exception e) {
        e.printStackTrace();
        throw new ConnectException("ERROR in doLoginPost(): "+e.getMessage());
    }
}

The server (which is not under my control) returns a 302 redirect when the login was successful, and 200 if it fails and re-loads the login page. When run with the above Jar files I get the 302 redirect, however if I run the exact same code from an Android project with the 1.6 Android Jar file on the build path I get the 200 response from the server. I get the same 200 response when running the code on my 2.2 device.

My android application has internet permissions, and the HttpGet works fine. I'm assuming that the problem lies in the fact that HttpPost (or some other class) is different in some significant way between the Android Jar version and the newer Apache versions.

I've tried adding the Apache libraries to the build path of the Android project, but due to the duplicate classes I get messages like: INFO/dalvikvm(390): DexOpt: not resolving ambiguous class 'Lorg/apache/http/impl/client/DefaultHttpClient;' in the log. I've also tried using a MultipartEntity instead of the UrlEncodedFormEntity but I get the same 200 result.

So, I have a few questions:
- Can I force the code running under android to use the newer Apache libraries in preference to the Android versions?
- If not, does anyone have any ideas how can I alter my code so that it works with the Android Jar?
- Are there any other, totally different approaches to doing an HttpPost in Android?
- Any other ideas?

I've read a lot of posts and code but I'm not getting anywhere. I've been stuck on this for a couple of days and I'm at a loss how to get the thing to work, so I'll try anything at this point. Thanks in advance.

Community
  • 1
  • 1
dave.c
  • 10,862
  • 5
  • 36
  • 61

4 Answers4

6

I have now given up on getting the HttpClient route to give the expected response from the server when run on Android. Instead I rewrote the doPost method above to use an HttpsURLConnection instead. Here's the new (working) version in the hope that it's useful to someone.

private static LoginBean altPost(String username, String password, String cookie, String token){
    LoginBean loginBean = new LoginBean();
    HttpsURLConnection urlc = null;
    OutputStreamWriter out = null;
    DataOutputStream dataout = null;
    BufferedReader in = null;
    try {
        URL url = new URL(URL_LOGIN_SUBMIT);
        urlc = (HttpsURLConnection) url.openConnection();
        urlc.setRequestMethod("POST");
        urlc.setDoOutput(true);
        urlc.setDoInput(true);
        urlc.setUseCaches(false);
        urlc.setAllowUserInteraction(false);
        urlc.setRequestProperty(HEADER_USER_AGENT, HEADER_USER_AGENT_VALUE_FF);
        urlc.setRequestProperty("Cookie", cookie);
        urlc.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
        String output = "org.apache.struts.taglib.html.TOKEN="+ URLEncoder.encode(token, HTTP.UTF_8)
                +"&showLogin=true&upgrade=&username="+ URLEncoder.encode(username, HTTP.UTF_8)
                +"&password="+ URLEncoder.encode(password, HTTP.UTF_8)+"&submit="
                +URLEncoder.encode("Secure+Log+in", HTTP.UTF_8);
        dataout = new DataOutputStream(urlc.getOutputStream());
        // perform POST operation
        dataout.writeBytes(output);
        // get response info
        loginBean.setResponseCode(urlc.getResponseCode());
        // get required headers
        String headerName = null;
        StringBuffer newCookie = new StringBuffer(100);
        String redirectLocation = "";
        for (int i=1; (headerName = urlc.getHeaderField(i)) != null;i++) {
            if (headerName.indexOf(COOKIE_VALUE_SESSION) > -1) {
                if (newCookie.length() > 0) {newCookie.append("; ");}
                newCookie.append(headerName);
            }
            if (headerName.indexOf(COOKIE_VALUE_AUTH) > -1) {
                if (newCookie.length() > 0) {newCookie.append("; ");}
                newCookie.append(headerName);
            }
            if (headerName.indexOf("https://") > -1) {
                redirectLocation = headerName;
            }
        }
        loginBean.setCookie(newCookie.toString());
        loginBean.setRedirectUrl(redirectLocation);

        in = new BufferedReader(new InputStreamReader(urlc.getInputStream()),8096);
        String response;
        // write html to System.out for debug
        while ((response = in.readLine()) != null) {
            System.out.println(response);
        }
        in.close();
    } catch (ProtocolException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
        if (out != null) {
            try {
                out.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        if (in != null) {
            try {
                in.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
    return loginBean;
}

I still have no idea why the HttpClient way didn't work properly.

dave.c
  • 10,862
  • 5
  • 36
  • 61
1

to avoid the collisions use this jar for httpclient

httplib

and this post would also be very useful

stack overflow post

Community
  • 1
  • 1
sunil shetty
  • 75
  • 1
  • 8
0

Is it possible that this website does user-agent detection and actually returns different results because it's Android? Given that 200 implies success, why must it give a 302 instead of a 200? Have you printed out the result that you get when it returns a 200, and does it give any additional information?

Michael Aaron Safyan
  • 87,518
  • 14
  • 130
  • 194
  • The 200 result is just the user login page again, with no error messages or other info, which is what normally happens with an incorrect login. I'm setting the user-agent explicitly in the headers, and it is set the same whichever way the code is called. I can login to the site OK from the browser on my android phone, which implies it is not blocking based on the fact its android. – dave.c Nov 19 '10 at 03:38
0

Check the RedirectHandler, override the default one and do some logging in it, I had problems with that when going to Android...

dacwe
  • 41,465
  • 11
  • 110
  • 135
  • I'm fairly happy with the redirect code (which I didn't include above), the problem is I'm not even getting that far. Since the server is returning a 200 code from Android rather than the 302 returned from Eclipse, I'm not getting to the part where I need to handle the redirect. – dave.c Nov 19 '10 at 12:54
  • Hum, could it be a user agent problem? – dacwe Nov 20 '10 at 16:53
  • The user agent is set in the above code, and is set to the same value whether it is run within Android or as pure Java. I've tried an android user agent, a FF/windows one and some custom ones. They all give the same behaviour. – dave.c Nov 23 '10 at 00:07