1

I want to parse a webpage called autocoder O*NET

so I want to send request to the URL, In an HTTP POST request, the parameters are not sent along with the URL as mentioned in HERE

My question is how I know the parameters I should pass? in autocoder o*net I consider the textfield in the form as a parameters that I should pass to the url is this right?

Map<String,Object> params = new LinkedHashMap<>();
    params.put("jobtitle", "Back-End Developer");
    params.put("jobdesc", "");
    params.put("educcode", "");
    params.put("naics", "");
    params.put("category", "");
    params.put("employer", "");
Community
  • 1
  • 1
Abeer
  • 81
  • 1
  • 1
  • 12
  • What's your intention to do? Do you want to parse the webpage by sending an HTTP GET Request and parsing the HTTP RESPONSE? Or do you want to send an HTTP POST request for sending data that shall be used by the called page as defaults for the text form fields? – Niklas P Oct 24 '16 at 14:24
  • I want to send a HTTP Post request for sure to get the data as if it sent by the text for fields – Abeer Oct 24 '16 at 14:27

2 Answers2

1

Put the params in the http request body.(I suggest you use Jsoup to do it.)

you can use the chrome debug mode, like this screenshot

IMXQD
  • 203
  • 1
  • 11
  • I don't know if you understand what I mean, but my question is how I know the parameters I should sent to url page that I don't know – Abeer Oct 24 '16 at 14:10
  • You can use the Google Chrome developer debug mode. – IMXQD Oct 24 '16 at 14:23
1

Unless the operator of the mentioned web site publishes an official api description for the url one can post to, you can only guess which parameters are meaningful. I haved used the firefox developers tools that are bundled with the browser (version 47.0.1) and found that these are likely to be processed:

  • action
  • view
  • jobtitle
  • jobdesc
  • codetype
  • employer
  • categroy
  • educcode
  • naics
Matthias
  • 3,228
  • 2
  • 23
  • 40
  • thank you this is very helpful, but when I run my code it give this exception `Server returned HTTP response code: 416` you know why? – Abeer Oct 24 '16 at 14:30