How to check File Integrity using md5 over camel route to S3?

Question

I am sending a file to s3 bucket using camel. I want to verify the file integrity using md5. i am using org.apache.commons.codec.digest.DigestUtils.

from(ftp_endpoint)
    .idempotentConsumer(simple("${in.header.CamelFileName}"), redisIdempotentRepository)
    .setHeader(S3Constants.KEY, simple("${file:name}"))
    .setHeader(S3Constants.CONTENT_MD5, simple(DigestUtils.md5(body().toString()).toString()))
    .to(s3_endpoint)

I am getting the following exception

com.amazonaws.services.s3.model.AmazonS3Exception: The Content-MD5 you specified was invalid. 
(Service: Amazon S3; Status Code: 400; Error Code: InvalidDigest; Request ID: 8462458C6250091C)

How do i calculate the MD5 correctly so that it uploads to S3.

score 1 · Answer 1 · edited May 23 '17 at 10:32

I can spot a couple of issues in your setHeader.

.setHeader(S3Constants.CONTENT_MD5, simple(DigestUtils.md5(body().toString()).toString()))

First, you are NOT calculating MD5 of your body (which I assume it's byte[] since you're reading a file) because you are calling toString() on it.
Second, docs for DigestUtils.md5 states that the return type is byte[] which once again you are calling toString() on it.

Calling toString() on a byte array returns a String containing something like

[B@106d69c

See for example this other question on SO "UTF-8 byte[] to String".

You can try this solution using DigestUtils.md5Hex which returns the hash as a String:

.setHeader(S3Constants.CONTENT_MD5, simple(DigestUtils.md5Hex(body())))

score 1 · Accepted Answer · answered Jun 16 '17 at 04:40

This works for me.

from(ftp_endpoint)
    .idempotentConsumer(simple("${in.header.CamelFileName}"), redisIdempotentRepository)
    .setHeader(S3Constants.KEY, simple("${file:name}"))
    .process(md5HeadersProcessor)
    .to(s3_endpoint)


public class Md5HeadersProcessor implements Processor {
    private java.util.Base64.Encoder encoder = java.util.Base64.getEncoder();

    @Override
    public void process(Exchange exchange) throws NoSuchAlgorithmException {
        byte[] bytes = exchange.getIn().getBody(byte[].class);
        MessageDigest md = MessageDigest.getInstance("MD5");
        md.update(bytes);
        String md5= encoder.encodeToString(md.digest());
        exchange.getIn().setHeader(S3Constants.CONTENT_MD5, md5);
    }
}

How to check File Integrity using md5 over camel route to S3?

2 Answers2