0

for example，grab full content from https://play.google.com/store/apps

I found it posts data:

"start=15&num=5&numChildren=10pagTok=CA8QDxjh2ND3psHQ4pcB%3AS%3AANO1ljLBy5U&ipf=1&xhr=1"

to show next browser page then I used

  HttpPost httpPost = new HttpPost("https://play.google.com/store/apps");
  List<NameValuePair> params = new ArrayList<NameValuePair>();
    params.add(new BasicNameValuePair("start","15"));
    params.add(new BasicNameValuePair("num","5"));
    params.add(new BasicNameValuePair("numChildren","10"));
    params.add(new BasicNameValuePair("pagTok","CA8QDxjh2ND3psHQ4pcB"));
    params.add(new BasicNameValuePair("ipf","1"));
    params.add(new BasicNameValuePair("xhr","1"));

    httpPost.setEntity(new UrlEncodedFormEntity(params, "UTF-8"));
    CloseableHttpResponse response = getSSLHttpClient().execute(httpPost);
    HttpEntity entity = response.getEntity();
    try {
        if(entity != null) {
            return EntityUtils.toString(entity);
        }
    } finally {
        EntityUtils.consume(entity);
        response.close();
    }

but in the end I can't get the web document from googleplay,the result is some javascript, what's wrong？

RTM
  • 73
  • 1
  • 9

1 Answers1

0

Apache HTTP Client is used to get the page content. If you need to execute javascript on this page you can use HtmlUnit. Here is an example

pomkine
  • 1,537
  • 4
  • 17
  • 29
  • I used HtmlUnit and http://stackoverflow.com/questions/12119610/crawl-dynamic-web-page-using-htmlunit ,didn't work for me – RTM Nov 25 '14 at 07:16
  • @RTM Some code would be nice. And btw check this out https://code.google.com/p/android-market-api/. Maybe it can help you. – pomkine Nov 25 '14 at 08:28
  • thank you,I solved the problem by changing param "start" and "num",I can get complete web content by posting "start=1&num=15&numChildren=10pagTok=CA8QDxjh2ND3psHQ4pcB%3AS%3AANO1ljLBy5U&ipf=1&xhr=1" – RTM Nov 26 '14 at 09:26