Google app engine: NullpointerException while storing image in Blobstore through a java servlet

Question

I'm trying to upload images via a JSP file upload and a validation servlet to the Blobstore. the JSP part is as follows:

<form action="/testuploadmimevalidation?provider-key=testprovider" method="post" enctype="multipart/form-data">
    <input type="text" name="foo">
    <input type="file" name="myfile" >
    <input type="submit" value="Submit">
</form>

The java class TestUploadMimeValidation is as follows:

public class TestUploadMimeValidation extends HttpServlet {

    private BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService();
    private static final Logger log = Logger.getLogger(TestUploadMimeValidation.class.getName());
    private static final boolean PRODUCTION_MODE = SystemProperty.environment.value() == SystemProperty.Environment.Value.Production;
    private static final String URL_PREFIX = PRODUCTION_MODE ? "" : "http://127.0.0.1:8080";

    public void doPost(HttpServletRequest req, HttpServletResponse res) throws ServletException, IOException {
        InputStream in = req.getInputStream();
        int formDataLength = req.getContentLength();
        byte dataBytes[] = new byte[formDataLength];
        int len;
        ByteArrayOutputStream bos = new ByteArrayOutputStream();

        while ((len = in.read(dataBytes, 0, formDataLength)) != -1)
            bos.write(dataBytes, 0, len);

        dataBytes = bos.toByteArray();
        String urlStr = URL_PREFIX + BlobstoreServiceFactory.getBlobstoreService().createUploadUrl("/testupload");
        URLFetchService urlFetch = URLFetchServiceFactory.getURLFetchService();
        HTTPRequest request = new HTTPRequest(new URL(urlStr), HTTPMethod.POST, FetchOptions.Builder.withDeadline(20.0));
        request.setHeader(new HTTPHeader("Content-Type", "multipart/form-data"));
        request.setPayload(dataBytes);

        System.out.println("step1");

        try {
            HTTPResponse response = urlFetch.fetch(request);
            System.out.println("step2");
        } catch (IOException e) {

        } catch (NullPointerException e) {

        }

        System.out.println("step3");
    }
}

This class uploads the image to the Blobstore just fine, but I get a NullPointerException. the stacktrace is as follows:

WARNING: /_ah/upload/ag5tOGJ5dXMtZGV2ZWxvcHIbCxIVX19CbG9iVXBsb2FkU2Vzc2lvbl9fGAIM
java.lang.NullPointerException
        at javax.mail.internet.MimeMultipart.writeTo(MimeMultipart.java:143)
        at com.google.appengine.api.blobstore.dev.UploadBlobServlet.handleUpload(UploadBlobServlet.java:180)
        at com.google.appengine.api.blobstore.dev.UploadBlobServlet.access$000(UploadBlobServlet.java:72)
        at com.google.appengine.api.blobstore.dev.UploadBlobServlet$1.run(UploadBlobServlet.java:101)
        at java.security.AccessController.doPrivileged(Native Method)
        at com.google.appengine.api.blobstore.dev.UploadBlobServlet.doPost(UploadBlobServlet.java:98)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
        at com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:58)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:122)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
        at com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:70)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at com.google.appengine.tools.development.JettyContainerService$ApiProxyHandler.handle(JettyContainerService.java:351)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
        at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
        at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

Because of this NPE my code in the TestUpload.java class does not get executed. The printlns come out fine. It prints "step3". The image gets stored into the Datastore. But I just can't get the code in the TestUpload to run. Do you have any idea what might be causing the exception? I've tried to mess with the multipart, but wasn't very succesful.

Any help with this issue would be really appreciated.

Are you sending an email after upload ? Why is the error coming from avax.mail — Kal, May 17 '11 at 17:47
your JSP can't possibly be working, it has to call the blobstoreservice.createUploadURL() as part of its action. — , May 17 '11 at 21:11
@ Jarrod. the blobstoreservice.createUploadURL() gets called in the TestUploadMimeValidation servlet. my Blob file gets stored in the blobstore when this servlet is done but i'm getting that nullpointer exception. it also does not forward to the testupload servlet (the one stated in the createUploadUrl(). line #14 of the TestUploadMimeValidation servlet. — dant3InTrouble, May 19 '11 at 12:42
@ Kal the mimemultipart lacks some type of mime definition that I need to set. i was wondering if any of you know which mime definition i need to set with the MimeMultipart class. i thought that request.setHeader(new HTTPHeader("Content-Type", "multipart/form-data"));would have been enough. — dant3InTrouble, May 19 '11 at 12:44

score 4 · Answer 1 · answered May 17 '11 at 18:40

4

I use

<form action="<%= blobstoreService.createUploadUrl("/catalog/actions/add") %>" method="post" enctype="multipart/form-data" >

answered May 17 '11 at 18:40

MBonell

41
3

i want to call the blobstoreService.createUploadUrl() in the servlet because i want to validate the size and content type of the blob first. if i use the blobstoreService.createUploadUrl in the form as you suggested the blob will be stored and i have to validate the size and mime type after i has already been stored in the blobstore. – dant3InTrouble May 19 '11 at 12:46

score 1 · Answer 2 · edited May 23 '17 at 12:30

First you can see how to do your POST request here: Using java.net.URLConnection to fire and handle HTTP requests

String param = "value";
File textFile = new File("/path/to/file.txt");
File binaryFile = new File("/path/to/file.bin");
String boundary = Long.toHexString(System.currentTimeMillis()); // Just
// generate some unique random value.
String CRLF = "\r\n"; // Line separator required by multipart/form-data.
URLConnection connection = new URL(url).openConnection();
connection.setDoOutput(true);
connection.setRequestProperty("Content-Type",
    "multipart/form-data; boundary=" + boundary);
PrintWriter writer = null;
try {
    OutputStream output = connection.getOutputStream();
    writer = new PrintWriter(new OutputStreamWriter(output, charset),
            true); // true = autoFlush, important!
    // Send normal param.
    writer.append("--" + boundary).append(CRLF);
    writer.append("Content-Disposition: form-data; name=\"param\"")
            .append(CRLF);
    writer.append("Content-Type: text/plain; charset=" + charset).append(CRLF);
    writer.append(CRLF);
    writer.append(param).append(CRLF).flush();
    // Send text file.
    writer.append("--" + boundary).append(CRLF);
    writer.append(
        "Content-Disposition: form-data; name=\"textFile\"; filename=\""
            + textFile.getName() + "\"").append(CRLF);
    writer.append("Content-Type: text/plain; charset=" + charset).append(CRLF);
    writer.append(CRLF).flush();
    BufferedReader reader = new BufferedReader(new InputStreamReader(
            new FileInputStream(textFile), charset));
    try {
        for (String line; (line = reader.readLine()) != null;) {
            writer.append(line).append(CRLF);
        }
    } finally {
        try { reader.close(); } catch (IOException logOrIgnore) {}
    }
    writer.flush();
    // Send binary file.
    writer.append("--" + boundary).append(CRLF);
    writer.append(
        "Content-Disposition: form-data; name=\"binaryFile\"; filename=\""
            + binaryFile.getName() + "\"").append(CRLF);
    writer.append(
        "Content-Type: "
            + URLConnection.guessContentTypeFromName(binaryFile
                .getName())).append(CRLF);
    writer.append("Content-Transfer-Encoding: binary").append(CRLF);
    writer.append(CRLF).flush();
    InputStream input = new FileInputStream(binaryFile);
    try {
        byte[] buffer = new byte[1024];
        for (int length = 0; (length = input.read(buffer)) > 0;) {
            output.write(buffer, 0, length);
        }
        output.flush(); // Important! Output cannot be closed. Close of
                        // writer will close output as well.
    } finally {
        try { input.close(); } catch (IOException logOrIgnore) {}
    }
    writer.append(CRLF).flush(); // CRLF is important! It indicates end
                                    // of binary boundary.
    // End of multipart/form-data.
    writer.append("--" + boundary + "--").append(CRLF);
} finally {
    if (writer != null) writer.close();
}

I tried with apache http-client library and it worked but only locally. Because of socket limitation we can't use it. https://developers.google.com/appengine/docs/java/sockets/#limitations-and-restrictions We are stucked to use: java.net.URL; java.net.URLConnection;

For images use binary from the link above. It was pain in the ass till I get working code :)

score 0 · Answer 3 · edited May 23 '17 at 12:23

TL;DR: For those who got this specfically weird error when trying to use AJAX to upload instead of the traditional form submission route, the solution is to NOT set the "Content-Type" header when creating the AJAX request. Upon removing this, it started working for me.

More info: Apparently, passing in a FormData object as the body for the AJAX request will automatically set the "Content-Type" header with "multipart/form-data", as well the required boundary string (which is what would be missing if we just set the content-type ourselves).

Additional info: This boundary string is, AFAIK, randomly generated, and we cannot define our own boundary via the "Content-Type" header (I've tried, and the boundary used in the body is still the randomly generated string, causing a heap space error). The AJAX request must be assigning the randomly generated boundary string in the header and setting our own content type is overwriting that header.

I got the clue from here: html form enctype on ajax (though jQuery related, the logic of setting the Content-type header is native webkit/browser logic)

Google app engine: NullpointerException while storing image in Blobstore through a java servlet

3 Answers3