HTTP POSTing in Java without libraries

Question

I would like to, without libraries, POST a file to a PHP script in Java.

This is what the HTML would look like if it actually had a form:

<form action="http://example.com/upload.php" method="post" enctype="multipart/form-data" name="input">
<input type="file" name="image" /><input type="submit" value="Upload file"  />

Then I can just grab it/all the information it comes with in PHP with $_FILES["image"].

If I had a file example.png (files will always be PNGs, if that's important) I wanted to upload, how would I do that?

I suppose the simplest way to implement this would be to make a function that takes a File, a URL String, and I guess another name String because we shouldn't assume we always want to POST with the name as image. It would POST the file to that URL under that name and return a String of what the URL returned upon assumed success.

I wanted to do it without libraries because I usually try to do things without first, because if I wanted to, I could skip almost everything I do in Java and generate GUIs and everything, but then you learn that much less. When that didn't work, I tried adding the HttpClient library, which for some reason Eclipse couldn't see. I figured I'd ask a question, and since you guys could probably help me either way, I'd ask to do it without a library. — , Mar 04 '12 at 01:06
I too like to limit the libraries that I use. First I want to see if I can do what I want with just the JDK. Then, if I can't find it there, then I look for libs or write it myself and add it to boon. :) — RickHigh, Nov 10 '13 at 19:41

score 0 · Answer 1 · edited May 23 '17 at 11:44

I just wrote this up in another post. :)

I just added this to Boon for someone else. Boon is small. And the HTTP utility class is relatively free of dependencies. If you have trouble cut/paste, let me know. I can make it into a single class with just the bits you need.

Let me break down the bits that you want.

http://richardhightower.github.io/site/Boon/Welcome.html

    String response = HTTP.postForm ( "http://localhost:9220/test",
            Collections.EMPTY_MAP,
            map("hI", (Object)"hi-mom", "image", new byte[] {1,2,3})
    );

https://github.com/RichardHightower/boon

Now you can do it in one method call. :)

:)

You can cut and past the code into your project. There is one utility class called HTTP that uses the standard URL class from Java.

Let me break that down for you. (You can grab it here btw: http://richardhightower.github.io/site/Boon/Welcome.html)

I added this to boon based on this post (Http Post both string and binary parameters in Java):

public static String postForm(final String url, final Map<String, ?> headers,
                                            final Map<String, Object> formData
)

The key here is encoding binary data:

    String response = HTTP.postForm ( "http://localhost:9220/test",
            Collections.EMPTY_MAP,
            map("hI", (Object)"hi-mom", "image", new byte[] {1,2,3})
    );

    boolean ok = true;
    ok |= response.startsWith ("hI=hi-mom&image=%01%02%03\n") ||
            die("encoding did not work");

The above is a test showing it works as I understand the spec.

The key is that it is turning "image", new byte[] {1,2,3} into image\u0000=%01%02%03.

BTW map is just a utility method that creates a map (listing at bottom).

The http server is just an echo.

    return Exceptions.tryIt(String.class, new Exceptions.TrialWithReturn<String>() {
        @Override
        public String tryIt() throws Exception {
            URLConnection connection;
            connection = doPostFormData(url, headers, formData);
            return extractResponseString(connection);
        }
    });

The magic happens in the doPostFormData:

private static URLConnection doPostFormData(String url, Map<String, ?> headers,
                                    Map<String, Object> formData
) throws IOException {
    HttpURLConnection connection;/* Handle output. */


    connection = (HttpURLConnection) new URL(url).openConnection();
    connection.setConnectTimeout(DEFAULT_TIMEOUT_SECONDS * 1000);

    connection.setDoOutput(true);

    connection.addRequestProperty ( "Content-Type", "application/x-www-form-urlencoded" );

    ByteBuf buf = ByteBuf.create ( 244 );



    final Set<String> keys = formData.keySet ();

    int index = 0;
    for ( String key : keys )  {

        Object value = formData.get ( key );

        if (index > 0) {
            buf.addByte ( '&' );
        }


        buf.addUrlEncoded (  key  );
        buf.addByte ( '=' );

        if ( ! ( value instanceof byte[] ) ) {
            buf.addUrlEncoded ( value.toString () );
        } else {
            buf.addUrlEncodedByteArray((byte[]) value);
        }
        index++;
    }


    manageContentTypeHeaders ( "application/x-www-form-urlencoded",
            StandardCharsets.UTF_8.name (), connection );

    manageHeaders(headers, connection);


    int len = buf.len ();
    IO.write(connection.getOutputStream(),
            new String(buf.readForRecycle (), 0, len, StandardCharsets.UTF_8), IO.DEFAULT_CHARSET);
    return connection;
}

Notice the call to addUrlEncodedByteArray you pass a byte array. Java works fine with URL encoding of strings. I could not find an easy way to encode a byte array so I just wrote it.

public void addUrlEncodedByteArray ( byte[] value ) {



    final byte[] encoded = new byte [2];

    for (int index = 0; index < value.length; index++) {
        int i = value[index];

        if ( i >= 'a' && i <= 'z' ) {
            this.addByte ( i );
        } else if ( i >= 'A' && i <= 'Z' ) {
            this.addByte ( i );
        } else if ( i >= '0' && i <= '9' ) {
            this.addByte ( i );
        } else if ( i == '_' || i == '-' || i == '.' || i == '*') {
            this.addByte ( i );
        } else if ( i == ' ') {
            this.addByte ( '+' );
        } else {
            encodeByteIntoTwoAsciiCharBytes(i, encoded);
            this.addByte ( '%' );
            this.addByte ( encoded [0] );
            this.addByte ( encoded [1] );
        }

    }
}

It is not the prettiest. But the unit tests work. I am sure you get the gist. It follows the spec and converts accordingly.

All data not in a certain range get encoded with %hexdigit hexdigit.

Then you just have these two methods to finish up the encoding:

/**
 * Turns a single nibble into an ascii HEX digit.
 *
 * @param nibble the nibble to encode.
 *
 * @return the encoded nibble (1/2 byte).
 */
protected static int encodeNibbleToHexAsciiCharByte( final int nibble ) {

    switch ( nibble ) {
        case 0x00:
        case 0x01:
        case 0x02:
        case 0x03:
        case 0x04:
        case 0x05:
        case 0x06:
        case 0x07:
        case 0x08:
        case 0x09:
            return nibble + 0x30; // 0x30('0') - 0x39('9')
        case 0x0A:
        case 0x0B:
        case 0x0C:
        case 0x0D:
        case 0x0E:
        case 0x0F:
            return nibble + 0x57; // 0x41('a') - 0x46('f')
        default:
            die("illegal nibble: " + nibble);
            return -1;
    }
}


/**
 * Turn a single bytes into two hex character representation.
 *
 * @param decoded the byte to encode.
 * @param encoded the array to which each encoded nibbles are now ascii hex representations.
 */
public static void encodeByteIntoTwoAsciiCharBytes(final int decoded, final byte[] encoded) {

    Objects.requireNonNull ( encoded );

    boolean ok = true;


    ok |= encoded.length == 2 || die("encoded array must be 2");


    encoded[0] = (byte) encodeNibbleToHexAsciiCharByte((decoded >> 4) & 0x0F);
    encoded[1] = (byte) encodeNibbleToHexAsciiCharByte(decoded & 0x0F);
}

That is the important bits. The rest is just dealing with HTTP request / header gak.

Here is manageContentTypeHeaders

    manageContentTypeHeaders ( "application/x-www-form-urlencoded",
            StandardCharsets.UTF_8.name (), connection );

...

private static void manageContentTypeHeaders(String contentType, String charset, URLConnection connection) {
    connection.setRequestProperty("Accept-Charset", charset == null ? StandardCharsets.UTF_8.displayName() : charset);
    if (contentType!=null && !contentType.isEmpty()) {
        connection.setRequestProperty("Content-Type", contentType);
    }
}

Here is manage headers

    manageHeaders(headers, connection);

...

private static void manageHeaders(Map<String, ?> headers, URLConnection connection) {
    if (headers != null) {
        for (Map.Entry<String, ?> entry : headers.entrySet()) {
            connection.setRequestProperty(entry.getKey(), entry.getValue().toString());
        }
    }
}

Then we encode the stream to send with UTF_8:

    int len = buf.len ();
    IO.write(connection.getOutputStream(),
            new String(buf.readForRecycle (), 0, len, StandardCharsets.UTF_8), IO.DEFAULT_CHARSET);

The IO write just does this: IO.write...

public static void write ( OutputStream out, String content, Charset charset ) {

    try ( OutputStream o = out ) {
        o.write ( content.getBytes ( charset ) );
    } catch ( Exception ex ) {
        Exceptions.handle ( ex );
    }

}

ByteBuf is just like a ByteBuffer but easier to use and very fast. I have benchmarks. :)

What did I miss?

Let me know if it works for you.

--Rick

The map function are just utility methods so I can concisely represent a map as I find I use them a lot. It only goes to 9 or ten. Beyond that I have a way to pass a list of entries.

public static <K, V> Map<K, V> map(K k0, V v0) {
    Map<K, V> map = new LinkedHashMap<>(10);
    map.put(k0, v0);
    return map;
}

public static <K, V> Map<K, V> map(K k0, V v0, K k1, V v1) {
    Map<K, V> map = new LinkedHashMap<>(10);
    map.put(k0, v0);
    map.put(k1, v1);
    return map;
}


public static <K, V> Map<K, V> map(K k0, V v0, K k1, V v1, K k2, V v2) {
    Map<K, V> map = new LinkedHashMap<>(10);
    map.put(k0, v0);
    map.put(k1, v1);
    map.put(k2, v2);
    return map;
}

public static <K, V> Map<K, V> map(K k0, V v0, K k1, V v1, K k2, V v2, K k3,
                                   V v3) {
    Map<K, V> map = new LinkedHashMap<>(10);
    map.put(k0, v0);
    map.put(k1, v1);
    map.put(k2, v2);
    map.put(k3, v3);
    return map;
}

public static <K, V> Map<K, V> map(K k0, V v0, K k1, V v1, K k2, V v2, K k3,
                                   V v3, K k4, V v4) {
    Map<K, V> map = new LinkedHashMap<>(10);
    map.put(k0, v0);
    map.put(k1, v1);
    map.put(k2, v2);
    map.put(k3, v3);
    map.put(k4, v4);
    return map;
}

public static <K, V> Map<K, V> map(K k0, V v0, K k1, V v1, K k2, V v2, K k3,
                                   V v3, K k4, V v4, K k5, V v5) {
    Map<K, V> map = new LinkedHashMap<>(10);
    map.put(k0, v0);
    map.put(k1, v1);
    map.put(k2, v2);
    map.put(k3, v3);
    map.put(k4, v4);
    map.put(k5, v5);
    return map;
}

public static <K, V> Map<K, V> map(K k0, V v0, K k1, V v1, K k2, V v2, K k3,
                                   V v3, K k4, V v4, K k5, V v5, K k6, V v6) {
    Map<K, V> map = new LinkedHashMap<>(10);
    map.put(k0, v0);
    map.put(k1, v1);
    map.put(k2, v2);
    map.put(k3, v3);
    map.put(k4, v4);
    map.put(k5, v5);
    map.put(k6, v6);
    return map;
}

public static <K, V> Map<K, V> map(K k0, V v0, K k1, V v1, K k2, V v2, K k3,
                                   V v3, K k4, V v4, K k5, V v5, K k6, V v6, K k7, V v7) {
    Map<K, V> map = new LinkedHashMap<>(10);
    map.put(k0, v0);
    map.put(k1, v1);
    map.put(k2, v2);
    map.put(k3, v3);
    map.put(k4, v4);
    map.put(k5, v5);
    map.put(k6, v6);
    map.put(k7, v7);
    return map;
}

public static <K, V> Map<K, V> map(K k0, V v0, K k1, V v1, K k2, V v2, K k3,
                                   V v3, K k4, V v4, K k5, V v5, K k6, V v6, K k7, V v7, K k8, V v8) {
    Map<K, V> map = new LinkedHashMap<>(10);
    map.put(k0, v0);
    map.put(k1, v1);
    map.put(k2, v2);
    map.put(k3, v3);
    map.put(k4, v4);
    map.put(k5, v5);
    map.put(k6, v6);
    map.put(k7, v7);
    map.put(k8, v8);
    return map;
}

public static <K, V> Map<K, V> map(K k0, V v0, K k1, V v1, K k2, V v2, K k3,
                                   V v3, K k4, V v4, K k5, V v5, K k6, V v6, K k7, V v7, K k8, V v8,
                                   K k9, V v9) {
    Map<K, V> map = new LinkedHashMap<>(10);
    map.put(k0, v0);
    map.put(k1, v1);
    map.put(k2, v2);
    map.put(k3, v3);
    map.put(k4, v4);
    map.put(k5, v5);
    map.put(k6, v6);
    map.put(k7, v7);
    map.put(k8, v8);
    map.put(k9, v9);
    return map;
}

score 0 · Answer 2 · answered Mar 04 '12 at 01:04

0

I realize you specified not using libraries but I would recommend Apache HttpComponents HttpClient.

One problem you will have is encoding your file to include in your post.

answered Mar 04 '12 at 01:04

Sarge

2,367
2
21
36

score 0 · Answer 3 · edited May 23 '17 at 12:07

To debug the request and response you can use a program like Fiddler to look at the body of the POST when the browser does it and when the java client is doing it. In your client set the system property java.net.useSystemProxies to true so it will use Fiddler when you're debugging.

If you encode your body as multipart then PHP will put the file in $_FILES. If you do not use multipart encoding then your Java code is much simpler but then your PHP code has to manually process the POST data and save the file on the server side. This would also be more efficient because encoding increases the amount of data that must be sent. I would lean toward not encoding and just sending the data because I believe the encoding will be a pain and I don't think the PHP code to manually process the POST data will be that tricky.

It looks like you can access $HTTP_RAW_POST_DATA to get the POST data in PHP. If the files you need to upload are very big then this will not work. There may be another way.

EDIT:

The accepted answer here shows how to create a multipart POST using Java: Using java.net.URLConnection to fire and handle HTTP requests

score -1 · Answer 4 · answered Mar 04 '12 at 01:11

Do you really mean Java or do you mean JavaScript? Since you mentioned Java I follow this one...

I think the simplest way would be to get the File into a well-formed string, encode it to something which is good for transport (like Base64 which is not very hard to self-implement if you dont want to use libraries) and then you could do a POST-Request by sending the correct HTTP-Request by yourself.

It's not that hard but it's too much to explain here at StackOverflow. I mean at least for sending those HTTP-Headers it is really recommended to use the java libraries.

HTTP POSTing in Java without libraries

4 Answers4